AD  682X58 


ATUS  Report  No.  21 


£0PY  NO.  5  9 


CRITERIA  FOR  EVALUATING  THE  EFFECTIVENESS  OF 
(  LIBRARY  OPERATIONS  AND  SERVICES 


(ASE  III:  RECOMMENDED  CRITERIA  AND  METHODS 
,  FOR  THEIR  UTILIZATION 


January  1969 


D  D  C 

“I — 1/ 

FEB  2  5  7959 

£><£lbu'E Thi 

B 


FINAL  REPORT  of  Contract  DA-2801 7- AMC -3483  (A) 
for  Picatinny  Arsenal,  Dover,  New  Jersey 


This  document  has  been  approved  for  public  release 
and  sale;  its  distribution  is  unlimited 


John  I .  Thompson  &  Company 
1118  22nd  Street,  N.  W. 
Washington,  D,  C.,  20037 


ATLIS  REPORT  NO.  21 


Criteria  for  Evaluating  The  Effectiveness  of 
library  Operations  and  Services 

PHASE  III:  Recommended  Criteria  and  Methods 
for  Their  Utilization 

By:  C.  J.  Wessel 

and 

K.  L.  Moore 

Date:  January,  1969 

FINAL  REPORT  of  Contract  DA-2801 7-AMC-3483 (A)  for 
Picatinny  Arsenal,  Dover,  New  Jersey 


This  document  has  been  approved  for  public  release  and  sale; 
its  distribution  is  unlimited. 


John  I.  Thompson  &  Company 
1118  22nd  Street,  N.W, 
Washington,  D.C.  20037 


ABSTRACT 


This  final  report  assesses  and  recommends  criteria  and  methods 
for  evaluating  the  performance  (effectiveness  and  efficiency)  of 
technical  library  operations  and  services.  These  criteria  and  methods 
include  those  identified  in  the  state-of-the-art  of  librarv  evaluation 
(existing  criteria  and  methods)  and  those  developed  bv  adoption  of 
criteria  from  the  state-of-the-art  of  "scientific  management" 

(candidate  criteria  and  methods). 

The  final  product  is  a  list  of  recommended  criteria  and  associated 
methods  of  implementing  them.  There  are  4  proposed  techniques.’ 

(1)  SCORE  Analysis  -  a  technique  to  measure  the  effectiveness  of  a 

service  and  the  associated  change  in  effectiveness  due  to  a 
change  in  operations  or  costs. 

v. 2 )  SCOUT  Analysis  -  a  technique  to  determine  the  optimum  balance 

between  operations  which  vields  maximum  effectiveness  within 

budget  constraints. 

(3)  CORE  Anal vs is  -  a  technique  to  derive  unit  cost  standards  for 
given  operations  which  produces  a  given  quality  of  output. 

(4)  GAME  Analysis  -  a  technique  to  eliminate  unnecessary  work  or 
excessive  delays;  to  arrange  work  in  the  best  order;  to 
standardize  usage  of  proper  work  methods,  and  to  develop 
time  standards  to  accomplish  essential  events. 
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PREFACE 


This  is  the  final  report  of  the  third  and  final  phase  of  the 
study  "Criteria  for  Evaluating  the  Effectiveness  of  Library  Operations 
and  Services,"  performed  under  Contract  DA-28017-AMC-3483(A)  for 
Picatinny  Arsenal,  Dover,  New  Jersey. 

The  first  report  (Phase  I:  Literature  Search  and  State-of-the-Art , 
ATLIS  Report  No.  10,  DDC  Document  No.  AD- 649  468)  summarised  the  findings 
of  a  search  in  the  library  and  management  sciences  literature  and  ident¬ 
ified  the  spectrum  of  management  techniques  to  oe  considered  for  the 
evaluation  of  library  effectiveness  and  efficiency. 

The  final  report  of  Phase  II  (Data  Gathering  and  Evaluation,  ATLIS 
Report  No.  19,  DDC  Document  No.  AD-676  188): 

(1)  summarized  data  and  information  collected  to  facilitate  the 
development  of  criteria  for  the  evaluation  of  library  efficiency 
and  effectiveness; 

(2)  summarized  the  findings  on  the  missions  and  objectives  of  Army 
Technical  Libraries  (A.T.L.'s); 

(3)  summarized  the  areas  in  which  adequate  standards  and  measures 
for  performance  are  feasible,  and 

(4)  presented  tentative  (existing  and  candidate)  criteria  and  proposed 
management  techniques  for  implementing  them. 

This  final  report  of  Phase  III  (Recommended  Criteria  and  Methods 
for  Their  Utilization,  ATLIS  Report  No.  21): 

(1)  examines  the  tentative  criteria  (existing  and  candidate): 

(2)  assesses  the  reliability  of  the  measurement  tools: 

(3)  presents  a  detailed  description  of  the  recommended  methods  of 
implementation  of  the  criteria,  and 

(4)  discusses  the  applicability  of  the  criteria  and  methods  as  bases 
for  establishing  adequate  standards  for  performance  evaluation  of 
A.T.L.'s. 


f 


TABLE  OF  CONTENTS 


PREFACE 


Page  No. 


I.  Introduction  . . . . . . .  1 

II.  Statement  of  Existing  Criteria . 4 

III.  Candidate  Criteria  and  Methods  of  Implementation .  9 

IV.  SCORE  Analysis — Detailed  Description... . 16 

V.  SCOUT  Analysis — Detailed  Description . 38 

VI.  CORE  Analysis — Detailed  Description .  53 

VII.  GAME  Analysis — Detailed  Description . 60 

VIII.  Application  and  Reliability  of  SCORE,  SCOUT,  CORE 

and  GAME  Analyses . 83 

IX.  Conclusions . 92 


! 


I 

i 

i 


r 


iv 


ILLUSTRATIONS 


Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 

Figure 


Page  No. 


1.  Search  Service  (Librarian  Conducts  Search) 

Equivocal  Expression  of  Equivocal  Need.. Case  Ilia 

Generalized  Model . . .  17 

2.  Search  Service — Case  Ilia,  January,  Year  "A" .  19 

3.  Search  Service — Case  Ilia,  February,  Year  "A" .  21 

4.  Search  Service-Case  Ilia,  March,  Year  "A" .  23 

5.  Search  Service — Case  Ilia,  January-March,  Year  "A" .  27 

6.  Search  Service — Case  Ilia,  May-Julv,  Year  "A" .  29 

7.  Cost-Ef fectiveness  Graph .  33 

8.  Search  Service— Case  Ilia,  Sept. -Nov.,  Year  "A" .  35 

9.  Total  and  Marginal  Utility  Change  Schedule .  39 

10.  Cataloging  Utility  Curve .  43 

11.  Collection  Si;  vs.  Percent  of  Needs  Covered .  4  7 

12.  Cataloging  Output  vs.  Cost  Scatter  Diagram .  34 

13.  Cataloging  Quality  Continuum .  5b 

14.  Cataloging  Method  Flow  Chart .  61 

15.  break  Even  Point  Chart .  78 

16.  Revised  Cataloging  Method  Flow  Chart . .  79 


v 


TABLES 


Pace  No. 


Table  1.  Utility  Value  Table . 49 

Table  2.  Cataloging  Output  vs.  Cost  Data  Sheet .  55 

Table  3.  Descriptive  Cataloging  Output  vs.  Cost  Data  Sheet .  56 

Table  4.  Cataloging  Time  Standard  Data  Sheet .  69 

Table  3.  Event  Occurrence  Data  Sheet .  84 

Table  6.  Event  Occurrence  and  Probabilitv  Data  Sheet .  84 


FORMS 


Form  1.  Operator's  SCORE  Analysis  Data  Sheet .  25 

Form  2.  GAME  Aralvsis  Operator's  Data  Sheet .  80 

Form  3.  GAME  Analysis  Work  Sampling  Studv  Form .  81 

Form  4.  GAME  Analvsis  Work  Sampling  Summary  Sheet . 82 


EXHIBITS 


Exhibit  1.  GAME  Analvsis  Operator's  Data  Sheet,  Sheet  1  of  1 .  63 

Exhibit  2.  GAME  Analvsis  Work  Sampling  Study  Form, 

Sheet  1  of  2 . . .  64 

Sheet  2  of  2 .  65 

Exhibit  3.  GAME  Analvsis  Work  Sampling  .Summary  Sheet, 

Sheet  l  of  1,  Oct.  2,  1968 .  67 

Exhibit  4.  GAME  Analysis  Work  Sampling  Summary  Sheet, 

Sheet  1  of  1,  Oct.  7,  1968 .  73 


vi 


ACKNOWLEDGEMENTS 


The  authors  acknowledge  with  thanks  the  kind  cooperation  of  the 
library  administrators  and  personnel  of  the  Army  Technical  Libraries 
visited  during  the  work  of  this  study.  We  also  wish  to  extend  our 
gratitude  to  the  management  personnel  of  the  Scientific  and  Technical 
Information  Center  at  Picatinny  Arsenal,  Dover,  New  Jersev,  for  their 
participation  in  the  several  helpful  discussions  held  during  the 
progress  of  work  in  this  investigation. 


vii 


DISCLAIMER 


The  findings  of  this  report  are  not  to  be  construed  as  an  official 
Department  of  the  Army  position. 
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I .  INTRODUCTION 


The  purpose  of  Phase  III  in  the  words  of  the  contract  is  as  follows: 

"Using  the  data  and  information  collected  in  Phase  I  and 
II,  tentative  criteria  for  the  evaluation  of  libraries 
will  be  set  up.  These  criteria  will  be  tested  and  validated 
against  a  sample  of  Armv  Technical  Libraries,  and  will  serve 
as  a  basis  for  establishing  adequate  standards  for  perform¬ 
ance  evaluation  of  Armv  Technical  Libraries.  Product  of 
this  phase  will  include  the  required  criteria  to  achieve 
the  objective  of  this  task,  and  a  detailed  description  of 
the  method  of  implementation." 

In  seeking  to  understand  the  requirements  of  Phase  III  so  chat 
maximum  results  may  be  realized  toward  fulfilling  the  objectives  of  the 
overall  study,  we  were  particularly  concerned  with  the  phrase:  "These 

criteria  will  be  tested  and  validated - ."  We  make  the  point,  in 

our  Phase  II  report  when  discussing  the  nature  of  criteria,  that  criteria 
comprise  oniy  one  part  of  a  whole  made  up  of  CRITERION-STANDARD-EFFECTIVE¬ 
NESS-EFFICIENCY.  That  is  to  say,  criteria  originate  out  of  knowledge  of  a 
subject,  are  of  a  broad  nature,  and  may  even  be  regarded  as  expressing 
truisms.  Puristically  they  are  important,  but  they  become  useful  only  when 
implemented  by  practical  methods.  Although  one  could,  of  course,  argue 
the  soundness  of  any  criteria  on  a  philosophical  level,  it  is  judged  more 
contributory  to  the  objectives  of  this  study  to  set  forth  the  criteria  as 
being  sound  and  valid  on  the  basis  of  knowledge  of  library  services, 
operations  and  management,  and  to  contend  that  what  is  needed  in  the  final 
analysis  are  tests  of  the  validity  and  reliability  cf  the  methods  by  which 
the  criteria  are  implemented.  It  is  bv  means  of  these  methods  that  we 
assess  the  criteria  and  finally  arrive  at  standards.  And  it  is  by  means 
of  standards  that  we  finally  achieve  effectiveness  and  efficiency.  This 
Interp-otation  was  discussed  with  the  technical  representative  of  the 
government  contractor  and  found  to  express  the  purpose  of  the  study. 

We  intend,  therefore,  in  this  report  to  sat  forth  the  criteria  which 
this  study  has  either  found  in  the  literature  or  which  we  have  developed 
in  the  course  of  the  work,  and  to  describe  in  detail  a  number  of  methods 
we  devised  for  implementing  these  criteria.  Our  discussions  will  describe 
how  the  suggested  methods  actually  may  be  used  to  implement  the  criteria, 
and  we  will  discuss  the  applicability  of  the  methods  and  criteria  to  the 
establishment  of  adequate  standards  for  evaluating  performance  of  A.T.L.'s. 

As  in  prior  cases*  we  believe  that  it  is  desirable  to  define  key 
terms  as  a  basis  of  subsequent  discussions.  Two  terms  of  considerable 
importance  to  this  work  are  "validity"  and  "reliability.  1  On  the  basis 

*  Definitions  for  the  terms  "criteria,"  "standard,"  "effectiveness"  and 
"efficiency"  have  been  provided  in  the  reports  of  earlier  phases. 
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of  definitions  in  authoritative  dictionaries  we  define  the  term  "valid," 
when  applied  to  criteria  or  methods,  as  meaning  that  these  criteria  and 
methods  are  sound  and  well-founded,  and  that  their  use  will  produce  the 
desired  result  of  being  able  to  measure  the  effectiveness  and/or  efficiency 
of  technical  libraries  in  a  meaningful  way.  Our  uses  of  the  words  "validity" 
and  "validate,''  therefore,  are  simply  as  noun  and  verb  forms  of  "valid." 

The  word  "reliability"  connotes  trustworthiness.  Again,  out  of  the 
common  definitions  of  authoritative  dictionaries  we  are  using  "reliability" 
as  meaning  that  the  methods  we  are  recommending  for  implementing  the 
criteria  are  dependable,  consistent  and  accurate,  and  will  truly  measure, 
with  uniform  results  when  repeatedly  applied,  the  effectiveness  of  library 
services  and  the  efficiency  of  library  operations. 

The  intimate  relationship  between  validity  and  reliability  sometimes 
causes  the  terms  to  be  regarded  as  synonymous.  We  wish  to  distinguish 
between  them,  however,  and  to  state  that  while  the  validity  of  the  criteria 
rests  upon  knowledge  of  the  field,  the  purpose  of  Phase  III  is  to  test  the 
validity  and  reliability  of  the  methods  for  applying  the  criteria. 

To  provide  continuity  with  the  reports  of  Phases  I  and  II,  a  few 
words  of  explanation  are  in  order.  As  will  be  recalled  from  the  earlier 
discussions,  it  was  stated  that  the  literature  search  of  Phase  I  found  a 
number  cf  statements  which  could  be  interpreted  as  criteria.  One  of  the 
tasks  of  Phase  II  was  to  review  the  findings  of  Phase  I  and  to  list  those 
statements  found  which  might  be  called  "existing  criteria."  In  the  Phase  II 
report  this  was  referred  to  as  the  TYPE  I  APPROACH.  Chapter  II  of  the 
present  report  will  list  that  group  of  criterion  statements  and  present 
findings  of  our  preliminary  discussions  of  them  with  several  library 
idministrators . 

In  addition  to  Type  1,  four  other  approaches  toward  developing 
criteria  were  described  in  the  Phase  II  report.  The  rationale  of  Type  II 
through  V  depends  upon  such  matters  as  mission  statements  of  A.T.L.'s; 
various  operational  data  concerning  these  libraries  gathered  by  other 
organizations  and  reported  in  a  series  of  referenced  documents;  the  ex¬ 
amination  of  modern  management  techniques  as  possible  methods  for  develop¬ 
ing  library  effectiveness  criteria,  with  subsequent  selection  of  a  limited 
number  of  such  techniques  as  most  applicable;  and  finally,  the  application 
of  i  modified  specific  management  technique  -  utility  analysis  -  to  a 
logical  development  of  library  services  and  operations  from  the  basic 
library  statements  of  mission,  goals  and  objectives.  From  Types  II 
through  V,  certain  concepts  were  suggested  as  tentative  criteria  in  the 
Phase  II  report.  These  will  be  referred  to  as  the  contract  "candidate 
criteria"  and  will  be  discussed  in  Chapter  III. 


The  work  of  Phase  III  resolved  itself  into  three  segments.  In 
segment  one,  visits  were  made  to  four  A.T.L.'s,  all  of  which  were  dif¬ 
ferent  from  the  10  libraries  visited  in  Phase  II,  to  gather  certain 
data  and  descriptive  information  needed  for  assessment  of  the  criteria 
and  methods.  In  addition  to  the  data  and  information  gathering  motive 
for  the  .library  visits,  we  wished  to  obtain  some  idea  of  the  informal 
acceptance  of  the  criteria  from  the  type  of  people,  namelv  the  library 
administrators,  who  eventually  would  be  expected  to  apply  the  criteria 
and  their  methods  of  implementation.  That  is  to  sav,  we  wanted  to  determine, 
in  a  preliminary  way,  whether  or  not  the  administrators  of  the  libraries 
visited  thought  tint  the  criterion  statements  are  sound  statements,  and 
whether  or  not  they  constitute  bases  on  which  to  develop  standards,  to 
measure  efficiency  and  to  measure  effectiveness. 

In  the  second  segment  of  work,  analysis  and  evaluation  of  the  state¬ 
ments,  data  and  descriptive  information  collected  were  made  according  to 
the  methods  developed  and  in  light  of  the  findings  in  Phases  I  and  II  to 
test  the  hypothesis  that  the  criteria  and  methods  constitute  a  practical 
basis  for  developing  standards.  In  the  third  segment,  the  material  was 
gathered  into  the  final  report  represented  by  this  document. 

Only  a  portion  of  the  data  and  information  gathered  was  actually 
utilized.  Not  all  possible  criteria  and  methods  initially  enunciated  were 
assessed.  Furthermore,  in  certain  of  the  explanations  and  descriptions  of 
methods  in  later  chapters,  hypothetical  data  are  used  simply  to  illustrate 
the  techniques. 
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II.  STATEMENT  OF  EXISTING  CRITERIA 


To  reiterate,  the  so-called  "existing  criteria"  have  been  derived 
from  the  literature  searched  in  Phase  I  and  from  statements  made  by 
librarians  during  the  Phase  II  library  visits.  These  criteria,  as  found, 
were  not  always  presented  by  the  originators  necessarily  as  "criteria" 
intended  to  be  the  basis  for  the  evaluation  of  library  performance. 

Some  of  them  were  stated  as  being  "policies,"  or  perhaps  "standards,"  or 
"rules."  Or  they  may  have  been  presented  simply  ac  principles  upon  which 
library  performance,  or  efficiency  or  effectiveness  might  be  based.  In 
certain  cases,  some  of  these  criteria  have  been  developed  to  a  rather  high 
degree  of  sophistication,  utilizing  measures  such  as  recall  and  relevance 
ratios.  But  most  of  them  are  general  statements  without  the  associated 
methodology  necessary  for  implementation,  and  without  any  suggested  basis 
for  quantification  or  measurement.  Nonetheless,  in  our  investigation  of 
the  matter  all  statements,  regardless  of  their  state  of  development, 
identified  as  being  existing  criteria  were  examined  by  our  ~taff  as  to 
their  usefulness  in  developing  methods  for  measuring  efficiency  and 
effectiveness. 

Thirty-seven  existing  criteria  were  identified,  drawn  together  in  a 
summary  form,  and  exhibited  to  the  administrators  of  the  four  libraries 
visited  in  Phase  III.  These  criteria  are  as  follows: 

1.  The  library  should  support  the  total  organizational  program 
and  goal. 

2.  The  library  goals  should  support  the  goals  of  the  parent 
organization. 

3.  The  library  goals  must  be  explicit  as  to  extent  of  service  and 
priority  requirements. 

4.  The  role  of  the  library  is  determined  largely  by  requirements 
for  library  services  of  the  laboratory  served. 

3.  A  technical  library's  mission  is  to  provide  technical  information. 

6.  A  college  library  should  be  directly  responsible  to  the  college 
president . 

7.  A  library  should  publicize  its  services  so  that  the  users  know 
thev  exist. 

R.  library  goals  must  be  agreed  upon  bv  management,  library 
administrator  and  users. 

9.  A  budget  for  a  special  library  should  he  based  on  recommendations 
of  the  library  administrator. 
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10.  The  library  administrator  should  have  the  responsibility 
and  authority  for  the  expenditure  of  his  budgeted  funds. 

11.  One  of  the  criteria  of  a  profession  is  the  existence  of  a 
systematic  body  of  knowledge  of  substantial  intellectual  content. 

12.  A  professional  should  have  personal  skill  in  the  application  of 
this  knowledge  (see  11)  to  solve  specific  problems  or  to  attain 
specific  goals  and  objectives. 

13.  An  adequate  professional  library  staff  is  an  essential  component 
of  basic  research  facilities. 

14.  The  library  staff  should  include  persons  trained  in  the  users' 
fields  who  are  familiar  with  information  problems  in  these  fields. 

15.  The  librarians  should  have  library  training  and  knowledge  of 
information  sources. 

16.  The  library  staff  is  responsible  for  the  efficient  organization 
of  materials  and  for  making  ■  ailable  the  catalogs  and  indexes 
for  prompt  access  to  the  materials  for  the  patrons. 

17.  A  job  description  should  include  a  statement  in  detail  of  actual 
activities,  indicating  importance  of  each  activity,  the  con¬ 
ditions  under  which  the  job  is  performed,  and  the  materials 
needed  to  carry  out  the  job. 

18.  Knowledge  requirements,  scope  of  assignment  and  level  of 
responsibility  should  be  factors  in  determining  grade  level  of 
personnel  in  a  government  library. 

19.  To  qualify  as  a  librarian,  a  person  should  have  five  years  of 
formal  education  beyond  secondary  school  including  graduation 
from  library  school. 

20.  A  special  library  should  acquire  materials  and  information  for 
the  current  and  future  needs  of  the  organization. 

21.  The  criteria  for  selection  of  published  material  should  inform 
the  acquisitioner  as  to  the  identity  of  the  material  that  should 
be  processed  and  also  should  indicate  to  the  clients  what  they 
can  expect  from  the  library. 

22.  The  organization's  information  requirements  should  reflect  all 
forms  of  information  required  to  fulfill  the  needs  of  research 
projects  and  technical  interests  of  group  members. 
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23.  Subject  coverage  of  the  collection  should  be  intensive  and  ^ 

extensive  enough  to  meet  the  current  and  anticipated  in¬ 
formation  requirements  of  the  library  clientele. 

24.  The  size  of  the  collection  should  depend  upon  the  amount  of 
material  available  that  is  pertinent  to  the  organization's 
needs . 

25.  The  quality  of  the  collection  can  be  judged  by  comparison  with 
lists  of  key  literature. 

26.  Relevance  ratios  are  a  measurement  of  effectiveness  of  indexing 
systems . 

27.  Recall  ratios  are  a  measurement  of  effectiveness  of  indexing 
systems . 

28.  The  catalog  should  be  in  enough  detail  for  the  users  to  be  able 
to  use  it  efficiently. 

29.  Cataloging  and  classifying  should  be  quality  controlled  in  such 
a  manner  as  to  assure  the  highest  possible  accuracy  and 
consistency. 

30.  Factors  to  be  considered  in  developing  standards  for  cataloging 

are:  ‘ 

a.  cost 

b.  physical  dimensions 

c.  time  to  construct 

d.  time  to  maintain 

e.  time  to  search 

f.  scope  of  topic 

g.  ease  in  determining  relevance 

h.  number  of  access  points  as  main  entry 

i.  universality  of  terms 

j.  rate  of  growth  per  new  entry 

k.  obsolescence  rate 

l.  simplicity  of  apparatus 

m.  adaptability  to  reproduction 

31.  Ease  in  accessibility  is  the  most  desirable  characteristic  in 
an  indexing  system. 

32.  Ease  in  accessibility  is  the  most  desirable  characteristic  in 
a  document  storage  system. 

33.  The  physical  location  of  materials  should  be  determined  bv  the 
amount  of  use. 
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34.  Success  in  locating  materials  is  more  significant  than  the 
number  of  volumes  in  the  collection. 

35.  The  amount  of  successful  use  of  a  library  is  the  ultimate 
test  of  its  effectiveness. 

36.  The  special  library  should  provide  successful  reference  service 
varying  from  answering  miscellaneous  questions  to  providing 
literature  surveys  and  comprehensive  bibliographic  reports. 

37.  The  library  staff  should  locate  library  materials  and  information 
promptly  upon  request. 

"Existing  criteria"  are  not  treated  in  this  study  as  candidate  criteria 
for  measurement  of  performance;  however,  they  are  used  in  many  cases  in 
the  development  of  candidate  criteria.  It  will  be  apparent  to  the  reader 
that  several  of  the  "existing  criteria"  are  similar  to  those  listed  dv  the 
authors  in  Chapter  III  of  this  report  as  "candidate  criteria."  For  example, 
numbers  1,  2,  3,  4,  20,  22  and  23  are  related  to  candidate  criterion  number  I 
of  Chapter  III.  Existing  criterion  number  30  enumerates  several  factors 
which  we  include  in  statements  of  candidate  criteria  or  which  we  develop  in 
methods  of  implementation.  Existing  criterion  number  35  is  a  basic  statement, 
the  implications  of  which  we  attempt  to  work  into  our  considerations  of  cost 
effectiveness  and  utility  analysis. 

We  are  interested  in  having  the  opinions  of  the  library  administrators 
on  the  following  four  points: 

(1)  Is  the  criterion  a  valid  statement? 

(2)  Is  the  criterion  a  basis  for  developing  standards? 

(3)  Is  the  criterion  a  basis  for  measurement  of  efficiency? 

(4)  Is  the  criterion  a  basis  for  measurement  of  effectiveness? 

Rather  than  report  all  detailed  information  which  resulted  from  our 
questions  to  the  library  administrators  on  the  existing  criteria,  it  is 
judged  sufficient  to  summarize  the  findings.  The  detailed  data  are  avail¬ 
able  should  they  be  required. 

The  majority  of  library  administrators  agreed  that  all  but  criteria 
numbers  6,  8  and  31  are  valid  statements.  None  of  the  existing  criteria 
received  majority  support  as  being  an  adequate  basis  for  measurement  of 
performance  (efficiency  and/or  effectiveness).  Among  other  areas,  there 
was  general  agreement  that  recall  and  relevance  measurements  taken  to¬ 
gether  may  be  adequate  measures  of  the  effectiveness  of  indexing  systems 
if,  and  only  if,  relevance  can  be  determined.  This  implies  that  client 
needs  must  be  defined  in  surficient  detail  so  that  relevance  can  be 
determined,  and  that  all  candidate  documents  can  be  classified  unequivocally 
as  being  either  relevant  or  irrelevant.  If  information  can  be  considered 
partially  relevant, or  relevant  to  anv  aspect  of  a  client's  need,  or  if 
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information  can  be  considered  as  being  potentially  useful  in  any  way  to 
serve  the  user's  expressed  need,  then  a  question  arises  as  to  degree  of 
relevance.  Furthermore,  if  the  need  is  equivocally  expressed,  relevance 
is  difficult  to  determine.  Further  research  should  be  conducted  to 
develop  criteria  for  determining  the  degree  of  relevance  of  information 
to  expressed  needs,  before  recall  and  relevance  ratios  can  serve  as  a 
measure  of  the  effectiveness  of  indexing  systems  adequate  for  the  develop¬ 
ment  of  standards. 

Other  existing  criteria  are  viewed  by  as  many  as  half  of  the  librarians 
queried  as  being  a  basis  for  measuring  efficiency  and/or  effectiveness. 
However,  the  majority  of  the  librarians  queried  feel  that  the  existing 
criteria  listed  have  not  been  developed  sufficiently  to  serve  as  a  basis 
for  developing  standards  for  operations  or  services.  Based  upon  this 
evaluation  of  "existing  criteria"  we  conclude  that  even  though  valid 
bases  for  measurement  of  some  aspects  of  effectiveness  have  been  developed, 
a  sufficient  number  of  factors  required  for  the  development  of  effectiveness 
standards  have  not  been  measured  adequately.  Some  of  the  A.T.L.'s  visited 
in  Phases  II  and  III  have  adequate  standards  for  measuring  efficiency; 
however,  the  criteria  and  methods  need  refinement  and  the  standards  developed 
are  not  universally  applicable.  We  have  incorporated  some  of  these  criteria 
and  methods  in  GAME  Analysis  (see  Chapter  VII)  —  a  technique  which  is 
universally  applicable  to  certain  operations  for  the  development  of 
standards . 

In  evaluating  f  "existing  criteria"  we  recognize  some  works  in  the 
state-of-the-art  of  i.>rarv  evaluation  as  being  stepping  stones  for  future 
development.  The  importance  of  this  background  should  not  be  taken  lightlv 
and  the  "existing  criteria"  should  be  exploited  in  areas  which  were  not 
developed  in  this  studv.  A  comprehensive  assessment  of  existing  criteria 
is  not  attempted  in  this  studv.  The  existing  criteria  useful  to  this 
studv  have  instead  been  incorporat’d  in  our  candidate  criteria  which  are 
assessed  in  the  following  chapter. 
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III.  CANDIDATE  CRITERIA  AND  METHODS  OF  IMPLEMENTATION 


Nine  basic  criteria  were  enunciated  out  of  the  work  of  Phases  I 
and  II  of  this  contract  study.  With  the  exception  of  one  of  these 
"candidate"  criteria,  methods  of  implementation  of  the  concepts  were 
also  recommended.  For  the  most  part  these  methods  may  be  identified  as 
"management  techniques."  Although  the  candidate  criteria  and  methods  were 
listed  in  the  Phase  II  report,  they  are  being  repeated  here  to  facilitate 
communication  in  this  document.  The  wording  and  arrangement  have  been 
revised  slightly  for  better  understanding,  but  the  essential  concepts 
remain  unchanged. 

CRITERION  I 


The  effectiveness  of  an  A.T.L.  is  a  function  of  the  extent  to  which  it 
supports  the  mission  of  the  parent  organization.  The  mission  cf  that 
library  is  a  derivative  of  the  mission  of  the  parent  organization.  The 
effectiveness  of  the  library  is  a  function  of  the  adequacy  and  clarity  of 
its  mission  statement  in  enumerating  concrete  goals  and  objectives  as  well 
as  specific  library  services  and  products. 

METHOD  1(a)  -  The  function  in  Criterion  I  can  be 
determined  by  organization  analysis,*  that  is,  bv  a 
studv  of  the  purpose  for  the  existence  of  the  organiza¬ 
tion,  its  function  and  the  extent  to  which  it  ac¬ 
complishes  its  functions. 

METHOD  1(b)  -  The  function  in  Criterion  I  can  be 
determined  by  research,*  that  is,  bv  a  studv  of  mission 
statements  of  other  libraries  with  similar  tvpes  of 
clientele,  to  develop  a  model  mission  statement. 

METHOD  1(c)  -  The  function  In  Criterion  I  can  be 
determined  bv  human  relations  studies,  that  is,  bv 
studies  of  customer  oriented  planning  and  control. 

METHOD  1(d)  -  Ttie  function  in  Criterion  I  can  be 
determined  bv  a  planning,  programming  and  budgeting 
svstem  studv,  that  is,  bv  a  studv  of  the  extent  to 
which  the  program  director  plans,  delineates  objectives, 
analyzes  costs  and  benefits,  and  allocates  his  budget 
to  maximize  benefits. 


The  management  techniques,  such  as  organization  analysis, "  "research" 
and  so  on,  referred  to  in  the  methods  statements,  are  defined  in  the 
glossary  of  the  Phase  II  report. 
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CRiraaoN  n  1 

The  effectiveness  of  an  A.T.L.  is  a  function  of  the  closeness  of 
Its  affiliation  with  the  administrative  level  resnonsiblc  for  the 
organizational  divisions  served  hv  the  library.  The  mission  of  the 
A.T.L.  is  approved  at  this  administrative  level  and  implemented  hy  the 
library  administrator. 

METHOD  IT (a)  -  The  function  in  Criterion  II  can  be 
determined  by  organization  analysis,  that  is,  by  a 
studv  which  Identifies  relationships  between  line  and 
staff  units  within  the  organization.  The  studv  i 
identifies  areas  of  responsibility  of  each  unit  and 
provides  a  view  of  the  distribution  of  responsibilities, 
thus  making  it  easier  to  detect  inconsistencies  of 
organizational  roles  and  goals. 

CRITERION  III 

The  effectiveness  of  an  A.T.L.  is  a  function  of  the  extent  to  which 
the  librarv  administrator  manages  his  resources  to  nrovide  the  combination 
of  services  and  products  which  give  optimum  support  to  the  library  mission, 
goals  and  objectives. 

METHOD  111(a)  -  The  function  in  Criterion  III  can  be 

determined  by  cost-effectiveness  analysis  and  by  a  j 

planning,  programming  and  budgeting  system  study,  that 

is,  by  utilizing  a  "systems  approach"  to  quantify  costs 

and  effectiveness  through  system  simulation  models.  The 

system  model  facilitates  mathematical  modeling  which  can 

be  the  basis  for  planning,  programming  and  budgeting 

decision. 

METHOD  III (b)  -  The  function  in  Criterion  III  can  be 
determined  by  utility  analysis,  that  is,  by  utilizing  the 
library  administrator's  judgments  as  to  the  value  of 
services  and  operations  in  supporting  the  library  mission. 

Basic  judgments  of  the  relative  values  of  each  service 
and  operation  are  recorded  in  such  a  way  that  mathematical 
equations  can  be  used  to  unify  them  and  to  resolve  the 
balance  between  operations  which  gives  optimum  support 
to  the  library  mission,  goals  and  objectives. 

CRITERION  IV 

The  effectiveness  of  a  given  type  of  service  or  product  is  a  function 
of  the  probabilities  of  occurrence  of  all  events  essential  to  meeting 
the  objective  or  ourpose  of  that  service  or  product. 
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METHOD  IV (a)  -  The  function  in  Criterion  IV  can  be 
determined  by  measuring,  fora  population  of  needs,  the 
percentage  of  needs  which  pass  each  event  required  to 
accomplish  the  objectives  of  the  services  or  to  ac¬ 
complish  the  production  of  the  required  products. 

CRITERION  V 


The  effectiveness  of  a  given  service  or  product  is  a  function  of 
the  collective  effectiveness  values  of  all  Individual  operations  in 
accomplishing  the  events  required  to  produce  it. 

NOTE:  No  method  is  offered  for  the  Implementation  of 
this  criterion  at  this  time. 

CRITERION  VI 


The  effectiveness  of  a  given  library  service  or  product  is  relative 
to  the  collective  indifference  (of  potential  users,  librarians,  and 
their  supervisors)  between  that  service  or  product  and  other  services  or 
products  needed  to  meet  their  respective  objectives. 

METHOD  VI (a)  -  The  relationship  in  Criterion  VI  can  be 
determined  by  subjective  analysis  of  the  value  or  utility 
of  each  service  or  product  in  supporting  the  mission  of 
the  library  and  the  mission  of  the  parent  organization, 
and  by  assigning  weights  to  reflect  the  relative  value  of 
each  service  or  product. 

CRITERION  VII 


The  efficiency  of  a  given  library  operation  is  a  function  of  the 
unit  cost  and  quality  (effectiveness)  of  the  operation's  outputs. 

METHOD  VII (a)  -  Where  high  correlations  exist  between 
operational  costs  and  outputs  within  and  among  A.T.L.'s, 
it  is  possible  to  develop  attainable  standards  of  ef¬ 
ficiency  for  given  ranges  of  effectiveness  on  the  basis 
of  these  correlations. 

METHOD  VI 1(b)  -  The  efficiency  of  routine  operations  can 
be  measured  against  standard  data  when  standard  units  of 
work  arc  produced.  Time  standards  for  routine  operations 
which  are  performed  to  accomplish  essential  events  in 
library  services  can  be  developed  for  each  routine  operation 
through  group  attainment  programs  or  other  work  sampling 
studies.  Group  efficiency  can  be  measured  against  the 
standard  and  expressed  as  an  index  of  staff  utilization. 


CRITERION  VIII 


The  effectiveness  of  a  given  operation  is  a  function  of  the  prob¬ 
ability  that  essential  events  occur  due  to  the  outputs  of  that  operation. 

METHOD  VIII (a)  -  The  function  in  Criterion  VIII  can  be 
determined  by  the  relationship  or  correlations  between 
certain  event  probabilities  and  the  outputs  of  operations. 

CRITERION  IX 


The  effectiveness  of  a  given  operation  is  a  function  of  its  output's 
contribution  to  the  total  value  of  the  library's  services  or  products. 

METHOD  IX (a)  -  The  function  in  Criterion  IX  can  be 
determined  by  analysis  of  the  value  or  utility  of  each 
operation's  output  in  adding  utility  to  various  services 
and  products. 

As  in  the  case  of  the  "existing"  criteria,  we  are  interested  in 
having  the  opinions  of  the  administrators  of  the  libraries  visited  with 
regard  to  the  "candidate"  criteria  and,  of  course,  the  methods  being 
proposed  for  their  implementation.  Therefore,  the  administrators  were 
asked  their  opinions  according  to  the  same  four  factors  referred  to  in 
the  prior  case,  that  is:  are  the  statements  sound,  and  do  they  appear  to 
constitute  a  basis  for  developing  standards,  for  measuring  efficiency 
and  for  measuring  effectiveness? 

The  opinions  expressed  by  the  library  administrators  were  based  in 
part  upon  data  which  we  collected  during  the  library  visits.  But  in  all 
cases  subjective  and  empirical  judgments  were  required.  The  data  col¬ 
lected  were  in  support  of  Criteria  III,  IV,  V,  VI,  VII,  VIII  and  IX. 

These  data  were  collected  when  needed  to  demonstrate  the  meanings  of  the 
criteria  to  the  librarians  participating. 

These  data  will  not  be  presented  in  this  chapter  due  to  their  large 
volume  and  complexity,  but  are  available  if  desired.  The  data  were  not 
collected  to  he  published  as  support  for  the  criteria,  but  rather  to  aid 
the  librarians  in  forming  an  opinion  as  to  the  usefulness  of  the  criteria 
as  a  basis  for  measuring  library  efficiency  and  effectiveness.  Some  of 
the  data  will  be  used  in  Chapter  VIII  to  demonstrate  the  reliability  of 
the  kinds  of  measurements  taken. 

All  of  the  candidate  criteria  were  considered  ns  being  valid  state¬ 
ments  by  most  of  the  librarians  queried.  The  usefulness  of  the  criteria, 
however,  will  depend  upon  the  particular  library  and  the  methods  of  im¬ 
plementation.  Chapter  VIII  will  discuss  the  reliability  of  the  methods 
and  will  discuss  the  feasibility  of  using  the  methods  for  developing 
standards  and/or  measuring  efficiency  and  effectiveness  at  different  types 
of  libraries. 


With  reference  to  the  methods  proposed  for  implementing  the 
candidate  criteria,  the  library  administrators  were  asked  if,  in  their 
oMnions,  the  methods  appear  to  constitute  a  sound  basis  for  measuring 
iciency  and/or  effectiveness  and  for  developing  standards.  They  were 
not  asked  their  opinions  as  to  whether  or  not  the  methods  are  reliable, 
for  this  remained  to  be  determined  by  the  contractor.  The  reliability 
of  the  methods  will  be  considered  in  Chapter  VIII. 

The  methods  supported  by  most  librarians  queried  as  being  useful  for 
development  of  standards  or  valid  for  measurement  of  efficiency  or 
effectiveness  are  111(a),  111(b),  IV(a) ,  VI(a),  VII(a),  VII(b),  VIII(a) 
and  IX(a).  These  methods  will  be  described  in  detail  in  Chapters  IV,  V, 

VI  and  VII.  Certain  other  methods  were  judged  as  promising,  but  were  not 
ronaidered  adequately  developed  for  determining  standards  or  for  measure¬ 
ment  of  performance  at  the  stage  of  their  development  as  presented  in  the 
Phase  II  report.  Methods  III  -  IV  and  VI  -  IX  will  be  combined  where 
possible  in  the  discussions  of  the  remainder  of  this  report  in  order  to 
reduce  the  number  of  methods  and  to  bring  them  together  to  form  tailored 
techniques  for  measurement  of  library  efficiency  and/or  effectiveness. 

Methods  III(s),  IV(a)  and  VIII(a)  arr  combined  to  form  a  technique 
which  we  shall  refer  to  as  SCORE  Analysis,  that  is.  Service  Components 
Reliability  and  Efficiency  Analysis.  The  Service  comprises  a  set  of 
operations,  equipment  and  materials  organized  to  meet  a  definable  user 
requirement,  whereas  the  components  of  tha  service  are  the  individual 
operations,  equipment  and  materials  which  are  necessary  to  provide  the 
service.  The  Reliability  of  the  components  is  derived  from  the  effect 
they  have  upon  the  probability  of  meeting  the  objectives  of  the  service. 

The  Efficiency  of  the  components  can  be  expressed  in  terms  of  their  cost 
and  effectiveness  probability  (reliability). 

SCORE  Analysis  determines  the  extent  to  which  the  library  administrator 
manages  his  resources  to  provide  the  combination  of  services  and  products 
which  gives  optimum  support  to  the  library  mission,  goals  and  objectives 
(see  Criterion  III).  SCORE  Analysis  makes  this  determination  by  [Method 
111(a)]  utilizing  a  "systems  approach"  to  quantify  costs  and  effective¬ 
ness  through  systems  (services)  simulation  models.* 

SCORE  Analysis  determines  the  probabilities  of  occurrence  of  all  events 
essential  to  satisfying  defined  service  objectives  (see  Criterion  IV) .  It 
makes  this  determination  by  measuring  [Method  IV(a)],  for  a  population  of 
needs,  that  percentage  which  passes  each  event  required  to  accomplish  the 
objectives  of  the  service. 


*  See  cost-effectiveness  analysis  in  Phase  II  glossary. 
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SCORE  Analysis  determines  the  probability  that  essential  events  will 
occur  due  to  the  outputs  of  operations  (see  Criterion  VIII) .  This  can 
be  determined  by  the  relationship  between  certain  event  probabilities  and 
*-he  outputs  of  operations  [Method  VI  11(a)  ] . 

Methods  111(b),  VI(a)  and  IX(a)  r, re  combined  to  form  a  technique 
which  we  shall  refer  to  as  SCOUT  Analysis,  that  is,  Service  Components 
UTility  Analysis.  SCOUT  Analysis  is  intended  to  measure  aspects  of  service, 
operations,  equipment  and  material  effectiveness  which  are  not  taken  into 
account  by  SCORE  Analysis.  SCOUT  Analysis  should  be  used  instead  of  SCORE 
Analysis  for  comparing  effectiveness  values  of  services  when  the  percentage 
of  needs  met  does  not  reflect  an  equitable  measure  of  relative  effective¬ 
ness  of  each  service.  That  is,  SCORE  Analysis  should  be  used  to  measure 
increases  in  effectiveness  of  a  service  and  the  associated  costs.  The  SCORE 
Analysis  effectiveness  measures  for  different  services  are  not  necessarily 
comparable  between  services.  To  use  an  analogy  a  score  of  100  points  in 
a  basketball  game  cannot  be  compared  to  a  score  of  10G  points  in  a  football 
game  to  reflect  the  same  level  of  attainment.  For  example,  a  library 
administrator  may  be  expected  to  judge  the  priority  of  clients  needs  and 
to  provide  adequate  service  according  to  priority.  If  this  is  the  case, 
the  extent  to  which  the  librarian  manages  his  resources  to  provide  the 
combination  of  services  which  give  optimum  support  to  the  library  mission 
(see  Criterion  III)  can  be  measured  according  to  the  librarian's  judgment 
of  the  value  of  each  service  in  supporting  the  mission.  In  SCOUT  Analysis, 
basic  judgments  of  the  relative  values  of  each  service  and  component 
(operation,  equipment  or  materials)  are  recorded  in  such  a  way  that  math¬ 
ematical  equations  can  ba  used  to  unify  them  and  to  resolve  the  balance 
between  various  operations,  as  well  as  equipment  and  materials,  which  gives 
optimum  support  to  the  library  mission  [see  Method  111(b)] .  By  this 
method  the  number  of  needs  met  in  each  service  can  be  weighted  (leveled) 
to  reflect  the  priority  of  services  and  consequently  to  derive  equitable 
measures  of  relative  value  (effectiveness)  of  each  service  and  component. 

The  effectiveness  of  a  given  library  service  or  product  is  relative 
to  the  collective  indifference  among  users,  potential  users,  librarians  and 
their  supervisors,  regarding  that  service  and  other  services  needed  to 
meet  their  res, ective  objectives  (see  Criterion  VI).  SCOUT  Analysis 
determines  these  relationships  by  subjective  analysis  of  the  value  or 
utility  of  each  service  in  supporting  the  mission  of  the  library  and  the 
parent  organization.  Weights  are  assigned  in  the  analysis  to  reflect 
the  collective  value  of  each  service  [see  Method  VI(a)].  SCOUT  Analysis 
departs  from  classical  utility  analysis  in  that  the  latter  requires  the 
registration  of  the  actual  and/or  potential  user  judgments  of  the  value 
of  a  commodity,  while  SCOUT  Analysis  does  not  necessarily  require  user 
value  judgments.  However,  SCOUT  Analysis  does  require  questioning  the 
users  and  the  potential  users  to  determine  the  value  of  the  available 
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services  given  by  the  library,  and  to  determine  the  values  of  varying 
degrees  and  kinds  of  services  in  supporting  the  mission  of  the  library 
and  thus,  ultimately,  the  mission  of  the  parent  organization. 

Since  a  military  reconnaissance  scout  becomes  adept  at  estimating 
the  numbers  and  strength  of  enemy  troops,  it  is  hypothesized  that  by 
similar  tactics  library  administrators  can  develop  skill  along  the  lines 
of  estimating  the  value  of  services  after  exposure  to  the  usage  made  of 
the  services  given  (i.e.,  by  observing,  looking  for  clues,  collecting 
data  and  interpreting).  This  hypothesis  was  supported  by  most  librarians 
queried  in  the  Phase  III  validation  process. 

The  effectiveness  of  a  given  operation  is  a  function  of  its  output 
contribution  to  the  total  value  of  library  services  (see  Criterion  IX) . 

SCOUT  Analysis  determines  the  value  or  utility  of  the  output  of  each 
operation  in  adding  utility  to  various  services  (see  Method  IX(a)]. 

Method  VII (a)  will  be  referred  to  as  CORE  Analysis,  that  is  Cor¬ 
relation,  Regression  and  Effectiveness  Analysis.  CORE  Analysis  can  be 
used  to  measure  the  efficiency  of  a  given  library  operation  as  a  function 
of  unit  cost  and  quality  (effectiveness)  of  operation's  outputs  (see 
Criterion  VII).  Where  high  correlations  exist  between  operational  output 
cost  and  quantity  for  given  quality  ranges  within  and  among  A.T.L.'s,  it 
is  possible  to  develop  attainable  standards  of  efficiency  by  regression 
analysis  for  given  ranges  of  effectiveness  [see  Method  VII(a)]. 

Method  VII (b)  will  be  referred  to  as  GAME  Analysis,  that  is,  Group 
Attainment  and  MEthods  Analysis.  GAME  Analysis  can  be  used  to  measure 
the  efficiency  of  a  given  library  operation  as  a  function  of  unit  cost 
and  quality  (effectiveness)  of  operation  outputs  (see  Criterion  VII) . 

GAME  Analysis  is  a  systematic  analysis  of  work  to  (1)  eliminate  unneces¬ 
sary  work  or  excessive  delays;  (2)  arrange  the  remaining  work  in  the 
best  possible  order;  (3)  standardize  usage  of  proper  work  methods,  and  (4) 
develop  time  standards  for  the  work  performed  to  accomplish  essential  events 
in  library  services.  Group  efficiency  can  be  measured  against  the  standard 
and  expressed  as  an  index  of  staff  utilization  as  in  the  group  attainment 
program  technique  (GAP).* 


See  GAP  and  Methods  Study  in  Appendix  A,  Phase  II  Report. 
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IV.  SCORE  ANALYSIS — DETAILED  DESCRIPTION* 


The  SCORE  Analysis  technique  will  be  described  in  detail  in  this 
chapter  for  a  simplified  sample  case.  The  case  at  point  will  be  an 
analysis  of  a  library  search  service.  In  this  case  the  user  requests 
information  on  a  given  subject  from  a  reference  librarian  (this  wil'  be 
referred  to  as  case  Ilia).  A  systems  model  simulating  this  service  is 
described  in  Figure  1.  Specific  case  models  such  as  this  one  can  be 
developed  for  each  library  service  case  and  are  based  on  the  generalized 
models  in  Chapter  VII  of  Phase  II.  For  purposes  of  simplicity  in  this 
description  we  shall  assume  that  the  library  performs  only  search  service 
(case  Ilia)  and  circulation  service.  We  shall  also  assume  that  the 
effectiveness  of  circulation  service  is  100%  and  that  the  library  is 
specialized  in  serving  research  and  development  only. 

The  effectiveness  of  the  library  and  search  service  (case  Ilia)  would 
then  be  the  product  of  probabilities  2,  3,  4,  6  and  7,  assuming  all  the 
needs  were  within  the  mission. 

TASK  1 


The  first  task  will  be  to  collect  statistics  on  the  number  of  needs 
processed  for  a  given  period  of  time,  e.g.,  1  month,  and  to  record  for 
each  need  the  data  required  in  the  Operator's  SCORE  Analysis  Data  Sheet 
(Form  1) . 

From  these  data  the  percentage  of  needs  which  passed  each  event  and 
the  labor  time  can  be  calculated  for  the  month.  If  for  the  month  of  January 
of  year  "A"  200  of  200  total  needs  were  communicated  adequately  to  Initiate 

a  search,  the  percentage  of  needs  passing  event  2  is  -.52.  X  100  -  100%. 

?.00 

During  the  same  month,  if  200  of  the  200  total  needs  entering  event  3  were 
considered  within  the  mission,  the  percentage  of  needs  passing  event  3  is 

X  100  =  100%.  If  180  of  the  200  needs  passed  event  4,  the  percentage 

of  needs  passing  event  4  *  X  100  ■  90%.  This  procedure  can  be  continued 

to  determine  the  percentage  of  needs  which  pass  each  event.  Figure  2  shows 
data  for  the  percentage  of  needs  met  by  each  event  and  cost  data  for  certain 
direct  labor  activities  during  January  for  the  hypothetical  special  library. 

The  same  data  are  collected  for  several  other  periods  of  time,  e.g.,  for 
February  and  March  of  year  "A",  and  the  percentage  of  needs  passing  each  event 
is  computed  and  recorded  in  Figures  3  and  4. 


SCORE  AnaWsis  Is  discussed  in  principle  in  Chapter  VII  of  the  Phase  II 
report  under  Cost-Effectiveness  Analysis  on  pp.65  through  73. 
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TASK  2 


Having  the  percentage  of  occurrence  of  each  event  for  3  periods, 
we  can  now  compute  the  probabilitv  of  event  occurrence.  Since  event  4 
percentages  (902,  922  and  88°;,  respectivelv ,  for  January,  February  and 
March)  do  not  range  significantly,  we  shall  consider  the  average  of  the 
3  percentages  as  the  probabilitv  of  event  4  occurrence  under  existing 
operating  conditions.  The  probability  of  event  4  occurrence  is,  therefore, 
.90. 


The  reliability  of  this  measure  and  other  measures  is  discussed  on 

p.  83  through  87  of  this  report. 

TASK  3 


The  next  task  is  to  compute  the  average  cost  per  need*  for  the  period. 
The  cost  per  need  for  activities  3-4  recorded  in  Figures  2,  3  and  4  were 
$.90,  $.94  and  $.87,  respectively,  for  January,  February  and  March.  The 
average  $.903  is  a  reliable  measure  of  the  cost  per  need  for  the  existing 
direct  labor  cost  of  conducting  searches,  since  the  range  in  average  cost 
for  the  3  periods  is  small. 

TASK  4 


Task  4  requires  changes  in  operational  procedures.  Figure  5  presents 
the  effectiveness  probabilities  and  the  cost  per  need  measures  for  the 
existing  service  under  existing  operating  conditions.  The  only  events 
which  prevent  1002  effectiveness  in  meeting  the  objective  of  providing 
some  relevant  documents  for  each  need  are  events  4  and  7.**  At  this  point 
changes  in  the  existing  operational  practices  or  policies  may  be  made  to 
attempt  to  increase  either  or  both  of  these  probabilities. 

It  is  not  the  primary  purpose  of  this  contract  to  develop  criteria 
to  direct  library  administrators  in  their  decisions  for  making  changes  in 
operational  policy  or  procedures;  instead  the  primary  purpose  of  this 
contract  and  SCORE  Analysis  is  to  provide  criteria  and  methods  for  measur¬ 
ing  operational  performance. 

The  changes  made  in  the  attempt  to  improve  effectiveness  should  be 
based  on  empirical  criteria  in  the  minds  of  the  administrators  themselves. 
Type  5  approach  discussed  in  the  Phase  II  report  may  help  to  formalize 
these  criteria  and  enable  the  administrator  to  manipulate  the  variables 
more  objectively.  By  whatever  means  employed,  we  will  assume  that  manage¬ 
ment  has  concluded  that  effectiveness  can  probably  be  improved  by  recom¬ 
munication  with  users  when  the  first  search  failed  to  identify  relevant 


Direct  labor  costs  are  derived  from  the  time  values  in  the  Operator's 
SCORE  Analysis  Data  Sheet  and  from  the  operator's  wage  rate. 

If  event  3  probabilitv  is  less  than  1002,  sub-probability  3a  (need  not 
within  the  scope  of  the  mission)  should  be  added  to  event  3  probabilitv 
for  computing  effectiveness. 


26 


SEARCH  SERV 
EQUIVOCAL 

JANU 


ENTER 


EVENT  I  -CASE  Ilia 


USER  CONTACTS  LIBRARIAN  TO 
SATISFY  INFORMATIONAL  NEED. 


200  NEEDS 
PER  MO. 

USER 

COMMUNICATES 
NEED  FOR 
REFERENCE  SERVICE 


Average  colt 
per  need 
S  .30 


EVENT  2 


USER’S  INFORMATIONAL  NEED 
IS  ADEQUATELY  COMMUNICATED 
TO  PROCESS  THE  NEED  FURTHER. 


PROBABILITY  —  t.00 


200  NEEDS 
PER  MO. 

LIBRARIAN 
REVIEWS  NEED 


Average  cent 
per  need 

S.03 


EVENT  3 


REFERENCE  LIBRARIAN 
CONSIDERS  CLIENT'S  NEED 
AS  WITHIN  THE  MISSION  OF 
THE  LIBRARY  AND  BEGINS 
SEARCH. 


PROBABILITY  —  1 .00 


REASON  FOR 
NOT  GIVING 
NEEDED  SERV¬ 
ICE 


Not  with¬ 
in  scope 
of  the 


mission 


b.  Priority 
of  other 
tasks 


SUB-PROBABILITY® 


SEARCH  SERVICE  (LIBRARIAN  CONDUCTS  SEARCH) 
EQUIVOCAL  EXPRESSION  OF  EQUIVOCAL  NEED 


CASE  Ilia 


JANUARY  -  MARCH.  Yf  AR  "A' 


EVENT  3 

REFERENCE  LIBRARIAN 

CONSIDERS  CLIENT'S  NEED 

AS  WITHIN  THE  MISSION  OF 

THE  LIBRARY  AND  BEGINS 

SEARCH. 

PROBABILITY  —  100 

REASON  FOR 
NOT  GIVING 
NEEDED  SERV¬ 
ICE 

SUB-PROBABILITY 

a.  Not  with¬ 
in  scope 
of  the 
mission 

0 

b.  Priority 
of  other 
tasks 

0 

200  NEEDS 

PER  MO. 

LIBRARIAN 

CONDUCTS 

REFERENCE 

SEARCH 

Average  cost 
per  need 

$.903 


EVENT  4  -  CASE  Ilia 

SOME  CANDIDATE  DOCUMENTS 

ARE  IDENTIFIED  OR  LOCATED 

BY  THE  LIBRARIAN. 

PROBABILITY  —  .90 

IDENTIFIED 

NO. 

SUB- 

ID. 

PROBABILITY 

a .  |n  cord 

1  * 

.90 

catalog 

2  - 

.85 

3  * 

.80 

4  - 

.70 

5  * 

.55 

6  * 

.40 

7  • 

.25 

8  • 

10 

9  f 

.02 

10  - 

.01 

b .  |n  other 

ref. 

0 

sources 

I  c.  By  information 

retrieval 

0 

system 

d .  In  another 

library 

• 

180  NEEDS 
PER  MO. 
LIBRARIAN 
AND  OR  USER 
RETRIEVES 
OR  ORDERS 
CANDIDATE 
DOCUMENTS 

Average  co»t 
per  need 
5.40 


EVENT  6 

SOME  OF  THF  CANDIDATE 
DOCUMENTS  ARF.  RE¬ 
TRIEVED  WITHIN  THE  RE¬ 
QUIRED  TIME. 

PROBABILITY  —  1.00 

ORDERED  FROM 

SUB-PROBABILITY 

a.  Stocks 

1.00 

b.  Inter- 
lihrory 
loa<' 

0 

c .  Vendor 

0 

180  NEEDS 
PER  MO. 

CLIENT  AND  OR 
LIBRARIAN  REVIEWS 
THE  RETRIEVED 
DOCUMENTS  TO 
DETERMINE 
RELEVANCE _ 

Average  cost 
per  need 

$.50 


INDIRECT  FACTORS 


FACTOR 

NO. 

COST 

No.  of  doc- 

200 

$5  25 

aments  in 

¥. 

PER 

collection 
per  subject 
dais 

requested 

NEED 

no.  of 

3 

1  SO 

index  terms 

PER 

per  title 
indexed 

NEED 

I 


DOCUMENTS 

LOCATED 


180  NEEDS 
PER  MO. 

librarian 
AND  /OR  USER 


SOME  OF  THE  CANDIDATE 
DOCUMENTS  ARE  RE¬ 
TRIEVED  WITHIN  THE  RE¬ 
QUIRED  TIME. 

PROBABILITY  —  1.00 


ORDERED  FROM  |  SUB-PROBABILITY 


180  NEEDS 
PER  MO. 

CLIENT  AND  OR 
LIBRARIAN  REVIEWS 
THE  RETRIEVED 
DOCUMENTS  TO 
DETERMINE 
RELEVANCE 


EVENT  7 -CASE  Ilia 


SOME  OF  THE  RETRIEVED 
DOCUMENTS  CONTAIN 
RELEVANT  INFORMATION. 
PROBABILITY  —  80 


SUB -PROBABILITY 


144  NEEDS 
PER  MO 
EXIT  TO 
CIRCULATION 
SERVICt 
ACTIVITY  7-8 


SEi 


221  NEEDS 
PER  MO. 

USE* 

COMMUNICATES 
NEED  FOR 
REFERENCE  SERVICE 


Average  co>i 
per  need 
J.2P 


EVENT  2 

USER’S  INFORMATIONAL  NEED 
IS  ADEQUATELY  COMMUNICATED 
TO  PROCESS  THE  NEED  FURTHER. 

PROBABILITY  —l.  00 


221  NEEDS 
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EVENT  3 

REFERENCE  LIBRARIAh 
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AS  WITHIN  THE  MISSI 
THE  LIBRARY  AND  BEC 
SEARCH. 

PROBABILITY  -  ,W6 


REASON  FOR 
NOT  GIVING 
NEEDED  SERV¬ 
ICE 

SUB-PR< 

a .  Not  with¬ 
in  i  cop* 
of  th* 
miuion 

i.  Priority 
of  othor 
talk* 

72  NEEDS 
PER  MO 

LIBRARIAN  COMMUNICATES  A  CONDU 
WITH  USER  (MUST  PASS  EVENT  3  FIRST! 


Average  c«>  per  need  S'  20 


SEARCH  SERVICE  (LIBRARIAN  CONOvXTS  SEARCH) 
EQUIVOCAL  EXPRESSION  Of  EQUIVOCAL  NEED 


CASE  Ilia 


MAY  -  JULY.  YEAR  "A'' 


REFERENCE  LIBRARIAN 
CONSIDERS  CLIENT'S  NEED 
AS  WITHIN  THE  MISSION  OF 
THE  LIBRARY  AND  BEGINS 
SEARCH. 

PROBABILITY  —  .996 


ASON  FOR 
5T  GIVING 
UDEO  SERV- 


t4ot  with¬ 
in  ICOpo 
Of  lNl 


Priority 
of  Othor 
tola 


SUB-PROBABILITY 


220  NEEDS 

PER  MO. 

LIBRARIAN 

CONDUCTS 

REFERENCE 

SEARCH 


IUNICATT  4 CONDUCTS  2nd  SEARCH 
PASS  EVINT  3  FIRST) 


$ 


EVENT  4  -  CaS£  !J!= 

SOME  CANDIDATE  DOCUMENTS 
ARE  IDENTIFIED  OR  LOCATED 
BY  THE  LIBRARIAN. 

PROBABILITY  -  .955 


IDENTIFIED 

o.  In  cord 
catalog 


210  NEEDS 

PER  MO. 

I -  LIBRARIAN 

su#-  AND  OR  USER 

PROBABILITY  retrieves 

! - Wi  OR  ORDERS 

’  VO  CANDIDATE 

R5  DOCUMENTS 


Avorago  co*t 
par  nood 
S.40 


b .  In  othor 


c  By  Inlormotion 
rofritvol 

lyttom 

d  In  anothor 
library 


SOME  OF  THE  CANDIDATE 
DOCUMEt  *S  ARE  RE¬ 
TRIEVED  WITHIN  THE  RE¬ 
QUIRED  TIME. 

PROBABILITY  -  1.00 

ORDERED  FROM  SUB-PROBABILITY 

o.  Stack*  1.00 

b.  Int»r- 
library 
loon 


210  NEEDS 
PER  mO 

CLIENT  AND  OR 
LIBRARIAN  REVIEWS 
THE  RETRIEVED 
DOCUMENTS  TO 
DETERMINE 
RELEVANCE 


i 


I 


4  -  CASE  Ilia 

ATE  DOCUMENTS 
DO*  LOCATED 
AN. 


SUB¬ 
PROBABILITY 
- ^ - 


110  NEEDS 
PE*  MO. 
LIMAftlAN 
AND  OR  USE* 
*ET«IEVES 
O*  O*0£*S 
CANDIDATE 
DOCUMENTS 


EV'  NT  6 

SOME  OF  THE  CANDIDATE 
DOCUMENTS  AKE  RE¬ 
TRIEVED  WITHIN  THE  RE¬ 
QUIRED  TIME. 

PROBABILITY  —  1.00 

ORDERED  FROM  I  SUB-PROBABILITY 


b.  Inttf- 
library 
loon 


210  NEEDS 

PE*  MO 

CLIENT  AND  OR 
LIBRARIAN  REVIEWS 
THE  RETRIEVED 
DOCUMENTS  TO 
DETERMINE 
RELEVANCE 


|  EVEN,'  7 -CASE  Ilia 

SOME  OF  THE  RETRIEVED 
DOCUMENTS  CONTAIN 
RELEVANT  INFORMATION. 
PROBABILITY  —  Ml 
'RETRIEVED  I  SUB -PROBABILITY 
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documents  anJ  bv  conducting  a  second  search.  Assume  that  additional 
man-hours  are  assigned  to  reference  activities  and  a  policy  is  written 
that  for  all  needs  which  are  not  provided  with  some  clearly  relevant 
documents,  a  second  communication  step  is  begun.  This  activity  will  have 
as  its  objective  (1)  a  better  definition  of  the  need,  (2)  a  more  ex¬ 
haustive  list  of  descriptors  and  (3)  more  specific  descriptors  and  their 
synonyms . 

The  new  policies  at  the  library  are  implemented  and,  after  a  period 
of  adjustment  to  the  new  operational  practic  s,  data  for  costs  and 
effectiveness  assessment  should  again  be  collected. 

TASK  3 

Task  5  is  a  cost-effectiveness  analysis  for  alternate  operational 
procedures.  Measures  of  percent  of  occurrence  of  events  and  cost  data 
are  compiled  for  the  new  procedures.  Let  us  assume  that  in  May,  June 
and  July  of  vear  "A"  the  rer'  munications  and  a  second  search  are  measured 
and  found  tc  take  an  average  of  50  minutes  for  each  need  which  was  not 
provided  wltn  some  relevant  documents  after  the  first  search,  and  the 
probabilities  of  events  4  and  7  increased  to  .955  and  .848,  respectively . 

All  other  costs  and  probabilities  remained  approximatelv  the  sane  as 
reflected  in  Figure  6 

From  the  data  in  Figures  5  and  6  we  can  compute  the  total  direct 
labor  cost  and  effectiveness  of  the  search  service  offered.  The  cost  for 
Januarv-March  is 

($.30) (200  needs)  +  ($.03) (200  needs)  +  ($.903) (200  needs) 

+  ($.40) (180  needs)  +  ($.50) (180  needs)  +  k  - 
$60.00  +  $6.00  +  $180.60  +  $72.00  +  $90.00  +  k  - 
$403.60  +  k 

where:  k  -  (constant  indirect  costs  +  overhead).*  The  effectiveness  for 
Januarv-March  is  the  product  of  the  event  probabilities 

(1 .00) (1 .00) ( .90) (1 .00) ( . 80)  -  .72 

The  total  cost  for  direct  labor  for  Mav-Julv  is 

( . 29) (221)  +  ( .03) (221)  ♦  ( . 90) (220)  ♦  (.40)(210)  ♦  (.53)(210) 

♦  (5.00) ( 22 >  +  k  - 

$64.09  *  $6.63  ♦  $198.00  ♦  $84.00  +  $111.30  +  $110.00  +  k  - 
$574.02  +  k 

and  the  effectiveness  is 

(1.00)(.996)(.955)(1.00)(.848)  -  .807. 

. . -nriT.  1.1,1-  '  -  .  »i  ■  *  ...  -  . .  i  ■!'  I  ■  l  ■  ■■  i  I  II 

k  is  a  constant  cost  figure  which  represents  costs  other  than  chose  which 
can  be  derived  from  the  operator’s  SCORE  Analvsis  data  sheet  and  wage  rates. 
These  'other'  costs  can  be  confuted,  but  it  will  not  be  necessary  to  do  so 
in  SCORE  Analysis  if  they  are  relatively  constant  costs  expended  primarily 
for  long-run  benefits. 
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We  now  have  cost  and  effectiveness  measures  for  these  2  periods.  However, 
they  are  not  comparable,  since  the  number  of  needs  met  during  the  2  periods 
vary.  To  have  comparable  figures  we  should  divide  the  cost  of  the  service 
by  the  number  of  needs  which  passed  event  7,  which  is  the  objective  of 
the  service.  The  comparable  figures  are: 

for  January-March 

.  $408. oO  +  k  _  $2.84  +  k 

0Sfc  144  needs  need 

Effectiveness  *  .720; 

for  May-Juiy 

_  $574.02  +  k  _  $3.22  +  k 
08  178  needs  need 

Effectiveness  *  .807* 

We  can  now  construct  a  cost-effectiveness  diagram  of  the  type  discussed  in 
the  Phase  II  report,  pp.  65  and  66.  The  cost-effectiveness  coordinate 
point  of  the  operational  procedures  for  January  through  March  is  represented 
by  the  pcint  marked  1  and  that  of  Mpv  ‘hrough  July  is  marked  2  (see  Figure  7). 
An  analysis  of  the  cost-ef fectivent  diagram  will  show  the  change  in 
effectiveness  with  respect  to  the  change  in  cost,  both  due  to  the  change 
in  communications  and  search  procedures.  This  relationship  is  mathematically 

represented  as  —  and  is  equal  to  *  .229.  In  Che 

AC  3.22  -  2.84 

denominator  we  drop  k  since  the  investment  of  k  adds  effectiveness  to  future 
needs.  We  shall  refer  to  this  quantity  (.229)  as  the  delta  index  of  cost- 
effectiveness  for  the  change.  Before  the  new  policies  and  procedures  are 
adopted  permanently,  the  library  administrator  should  compute  the  delta 
index  of  cost-effectiveness  for  all  other  feasible  alternate  solutions.  One 
possible  alternate  to  recommunication  and  a  second  search  may  be  to  refer 
all  needs  not  met  after  the  original  search  to  other  libraries  or  automated 
information  retrieval  certers  such  as  Defense  Documentation  Center.  Assume 
ror  the  months  of  September,  October  and  November  of  year  "A"  that  the 
library  did  not  recommunicate  with  users  or  conduct  a  second  search  when 
the  first  search  was  not  successful,  but  instead  requested  bibliographies 
from  selected  interlibrary  loan  and  information  retrieval  organizations. 

This  new  procedure  resolves  new  probabilities  and  costs  as  reflected  in 
Figure  8. 


It  is  possible  to  derive  a  negative  delta  index  of  cost-effectiveness; 
if  AE  is  negative  when  A  C  is  positive,  the  change  should  not  be  made; 
if  AC  is  negative  when  A  E  is  positive,  the  change  should  be  made 
permanent  if  the  budget  is  not  exceeded  and  if  a  better  alternate  solution 
is  not  found. 
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60  80 
PERCENT  EFFECTIVENESS 


Figure  7 


The  effectiveness  probability  of  the  Septenher-Sovember  operational 
procedure  can  be  computed  by  the  formula* 

number  of  satisfied  outgoing  needs 
number  of  incomine  needs 

or  for  the  operational  procedures  of  September  through  November  the 

186 

effectiveness  probability  is  =  .81. 

JU 

The  direct  cost  per  need  for  September  through  November  is 
(230) (.29)+(230)(.03)+(230) ( .93)+(207) ( .40)+(219) ( .52)+(13) (6.30) 

186 

66.70  +  6.90  +  213.90  +  82.80  +  133.88  +  31.90 

186 


*  566.08  *  S3. 05. 
186 


The  delta  index  of  cost-effectiveness  for  the  operational  procedures 

.81  -  .72  .09 

.21 


of  September  through  November  is 


.428, 


$3.05-$2.84 

relative  to  the  operational  procedural  index  for  January  through  March. 
The  delta  index  for  operational  procedures  for  Mav  through  June  vas  .229. 


After  Task  5  is  completed  measures  of  cost-effectiveness  of  feasible 
alternative  operational  procedures  are  available  and  SCORE  Analysis  is 
finished.  The  following  section  "SCORE  Anaivsis  and  P.P.B.S."  describes 
how  these  measures  may  be  used  to  facilitate  programming,  planning  and 
budgeting. 


SCORE  ANALYSIS  AND  P.P.B.S. 


Since  funds  should  be  allocated  to  increase  effectiveness  at  the 
least  cost  per  unit  of  effectiveness  gained,  the  funds  should  be  allocated 
to  resolve  the  highest  delta  index  of  cost-effectiveness.  Therefore,  the 
library  administrator  in  the  sample  case  might  adopt  the  policy  of  referring 
all  needs  not  met  by  the  original  search  to  interlibrary  loan  and/or  to  an 
outside  automated  information  retrieval  service.  If  additional  funds  are 
available  after  this  change,  perhaps  some  recommunications  and  some  second 
searches  should  also  become  part  of  the  procedures. 

In  a  case  where  no  additional  funds  are  allocated  it  may  be  necessary 
to  exchange  budget  allocations  between  services  or  operations.  For  example, 
with  a  policy  to  reduce  cost  of  cataloging  by  using  Lib-ary  of  Congress 
proof  slips  as  cataloging  information  and  to  add  ihe  funds  saved  to  inter- 
library  loan  operations,  the  index  of  cost-effectiveness  may  be  increased 
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This  formula  can  be  used  only  when  ail  needs  are  within  the  mission 
of  the  library  and  the  objectives  of  the  service. 


EVENT  2 


USER'S  INFORMATIONAL  NEED 
IS  ADEQUATELY  COMMUNICATED 
TO  PROCESS  THE  NEED  FURTHER. 


PROBABILITY  —  100 


230  NEEDS 
PER  MO. 

LIBRARIAN 
REVIEWS  NEED 


Average  cart 
per  need 
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EVENT  3  j 

REFERENCE  LIBRARIAN! 
CONSIDERS  CLIENT'S  I 
AS  WITHIN  THE  MISSIC 
THE  LIBRARY  AND  BEES 
SEARCH. 

PROBABILITY  —  1-00 

REASON  FOR  SUB-PRC 
NOT  GIVING 
NEEDED  SERV¬ 
ICE 


SEARCH  SERVICE  (LIBRARIAN  CONDUCTS  SEARCH) 
EQUIVOCAL  EXPRESSION  OF  EQUIVOCAL  NEED 


CASE  Ilia 


SEPT.  -  NOV.,  YEAR  "A' 


/ c< 

j-o  <r  /< 
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EVENT  5 

A  BIBLIOGRAPHY  IS  PROVIDED 
AND  USER  REVIEWS  BIBLIOG¬ 
RAPHY,  AND  IDENTIFIES 
SOME  CANDIDATE  DOCUMENTS 
FOR  RETRIEVAL. 

PROBABILITY  —  .928 


Wv* 

p\V) 

Ay/o0 
a V^*  t>  a 

* 


23L  needs 
PER  MO. 

LIBRARIAN 
REVIEWS  NEED 

Average  cor' 
per  need 

$.03 


EVENT  3 

REFERENCE  LIBRARIAN 

CONSIDERS  CLIENT'S  NEED 

AS  WITHIN  THE  MISSION  OF 

THE  LIBRARY  AND  BEGINS 

SEARCH. 

PROBABILITY  —  1.00 

REASON  FOR 
NOT  GIVING 
NEEDED  SERV¬ 
ICE 

SUB-PROBABILITY 

a.  Not  with¬ 
in  scope 
of  the 
mission 

o 

b.  Priority 
of  other 
tasks 

0 

230  NEEDS 
PER  MO. 

LIBRARIAN 

CONDUCTS 

REFERENCE 

SEARCH 

Averoge  cost 
per  need 

$.93 


EVENT  4  -  CASE  Ilia 

SOME  CANDIDATE  DOCUMENTS 
ARE  IDENTIFIED  OR  LOCATED 
BY  THE  LIBRARIAN. 

PROBABILITY  —  .966 


IDENTIFIED 

NO. 

ID. 

SUB¬ 

PROBABILITY 

O  -  In  cord 

1  r 

.90 

catalog 

2 

.85 

r  J  ‘ 

.80 

4 

.70 

5  ■ 

.55 

6  * 

.40 

2  . 

.25 

8  - 

.10 

9  f 

.02 

10  - 

.01 

b .  In  other 

ref. 

0 

sources 

c.  By  Information 
retrieval 
system 

d .  In  another 
library 
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DOCUMENTS 
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SOME  OF  THE  CANDIDATE 
DOCUMENTS  ARE  RE¬ 
TRIEVED  WITHIN  THE  RE¬ 
QUIRED  TIME. 
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ORDERED  FROM  SUB-PROBABILITY 

a.  Stocks  1.00 

b.  Inter- 
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DETERMINE 
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FOR  RETRIEVAL. 

PROBABILITY  —  .928 
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within  the  same  budget.  The  index  of  cost-effectiveness  referred  to 
here  is  the  total  effectiveness  of  the  service  divided  by  the  average 
total  cost  per  need  * 

In  a  case  where  funds  are  cut,  allocations  should  be  cut  on  operations 
and/or  services  to  resolve  the  highest  index  of  cost-effectiveness  for 
total  services. 

In  these  cases  we  see  the  operative  criteria  of  Programming-Planning 
and  Budgeting  Systems.  P.P.B.S.  uses  these  criteria  to  plan  further 
into  the  future  by  extrapolating  the  existing  trends  and  incorporating 
policies  which  can  be  predicted  to  resolve  optimum  cost-effectiveness. 

If  the  predictions  are  based  on  reliable  data  and  valid  criteria,  performance 
can  be  controlled  optimally  by  the  budgeting  process.  For  example,  if 
measurements  indicate  that  the  index  of  cost-effectiveness  is  improved  by 
using  Library  of  Congress  cataloging  information  to  facilitate  cataloging 
and  by  freeing  some  catalogers  to  assist  reference  librarians,  a  budget  can 
be  implemen" ed  to  control  the  phasing  in  of  the  new  operational  procedures 
and  lolicies  and  the  allocation  of  manpower  and  other  resources  to  catalog¬ 
ing  and  reference  operations. 

The  budget  should  be  structured  to  attain  a  goal.  Thus  the  goal  for 
a  five-vear  program  for  search  service  may  be  to  attain  at  the  5th  year 
an  effectiveness  of  .90  at  a  total  cost  of  less  than  $8.00  (+  inflationary 
factor)  per  need.  The  number  and  kinds  of  needs  entering  the  library  per 
month  should  be  projected  for  the  five-year  period.  With  the  goal  and 
the  expected  number  of  needs  given,  it  then  becomes  possible  to  anticipate 
budget  requirements  for  the  service  to  meet  the  five-year  goal.  The  P.P.B. 
System  will  not  be  described  in  detail  in  this  report,  since  the  primary 
purpose  of  this  contract  is  to  develop  detailed  methods  of  measuring 
performance.  The  P.P.B.  System  has  been  described  briefly  to  indicate 
how  the  SCORE  measures  mav  be  used  in  a  P.P.B.  System  to  improve  and 
control  performance. 

SCORE  Analysis  can  be  used  most  appropriately  to  improve  the  per¬ 
centage  of  needs  met  in  given  services.  If  some  services  vield  more 
utilitv  per  need  met  than  some  other  services,  SCORE  Analysis  should  not 
be  used  to  improve  the  percentage  of  needs  met  in  the  services  where  utilitv 
per  need  is  low  at  the  expense  of  services  where  the  utilitv  per  need  is 
high.  The  utility  concept  is  considered  further  in  the  following  chapter 
on  SCOUT  Analysis. 


V.  SCOUT  ANALYSIS— DETAILED  DESCRIPTION* 


The  usage  of  SCOUT  Analysis  will  be  described  in  detail  for  a 
hypothetical  A.T.L.  Figure  9  represents  the  total  and  marginal  utility 
schedule,  as  discussed  in  Chapter  VIII  of  the  Phase  II  report  with  some 
revisions.  The  following  sections  detail  the  tasks  to  be  accomplished 
in  SCOUT  Analysis. 

TASK  1 


The  first  task  to  be  accomplished  in  SCOUT  Analysis  is  to  list  the 
services  or  products  produced  by  the  library  (see  1,  Figure  9,  upper 
left  hand  comer).  The  services  and  products  given  in  the  hypothetical 
A.T.L.  are  Reference  Search  and  Circulation,  Circulation  upon  Request 
(no  previous  search),  Circulation  Predetermined,  and  Users  Search  and 
Circulation. 


TASK  2 


Task  2  lists  all  significant  operations  performed  in  the  library 
(see  2,  2nd  column  from  the  left,  Figure  9).  'I ne  operations  are  listed 
and  given  symbols  (see  symbols  A,  B,  C,.,.H)  in  the  first  column  at  the 
left. 

TASK  3 

Task  3  lists  the  significant  materials  used  by  the  library  in  giving 
its  services  or  products  (see  3  in  lower  left  comer).  The  only  sig¬ 
nificant  materials  used  at  the  hypothetical  A.T.L.  are  books  and  documents. 
The  symbol  which  represents  these  books  and  docun.  nts  is  I.  The  operations 
and  books  and  documents  are  the  only  significant  components  of  the  services 
given  at  the  hypothetical  A.T.L.  That  is,  the  significant  costs  in 
operating  the  library  are  assignable  to  these  components,  except  for 
management  operations  and  other  overhead  components  which  will  not  be 
considered  in  SCOUT  Analysis . 

TASK  A 


i 

1 


! 


Task  A  lists  the  average  existing  man-hours  assigned  per  week  to  each 
operation  in  the  3rd  column  from  the  left  (see  column  marked  A  under¬ 
lined  quantities)  and  incremental  deviations  both  above  and  below  the 
existing  man-hours  (see  input  man-hours  in  column  marked  A  not  underlined) . 
The  total  of  the  average  existing  man-hours  per  week  should  be  the  total 
man-hours  on  the  job  (that  is,  man-hours  paid  for)  for  the  average  week 
except  for  management  operations. 


Utility  Analysis  is  discussed  in  principle  in  Chapter  VIII  of  the 
Phase  II  report,  p.87  through  p.9A.  SCOUT  Analysis  is  an  outgrowth 
of  these  principles. 
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TASK  5 


Task  5  lists  the  quantity  of  materials  and  equipment  which  represent 
a  significant  investment,  such  as  the  books  and  documents  collection, 
computer  equipment  and  photographic  equipment.  For  the  purposes  of  this 
description  we  shall  consider  the  hypothetical  library  as  having  only  one 
significant  investment  in  materials,  that  is,  the  books  and  documents 
collection.  The  collection  size  is  then  recorded  in  the  quantity  column 
(see  column  marked  3  lower  left,  Figure  9). 

TASK  6 

The  sixth  task  assigns  weights  to  each  significant  existing  service 
or  product  given  bv  the  library.  These  weights  represent  the  existing 
relative  utility  of  each  service  or  product  in  meeting  the  wi thin-mission 
needs  of  the  users.  In  order  to  derive  equitable  weights  for  the  services 
and  products  the  librarian  must  scout  the  services  and  products  given  to 
determine  (1)  the  kinds  and  degrees  of  each  service  and  product  given, 
and  (2)  the  effect  of  the  information  derived  from  the  respective  service 
or  product  in  meeting  a  specific  within-mission  need  or  in  leading  the 
user  to  a  potential  approach  for  meeting  his  within-mission  goals.  It 
then  beccmes  possible  to  weight  the  value  or  utilitv  of  each  service  and 
product.  A  number  of  different  mechanisms  can  be  used  to  facilitate  the 
weighting  exercise.  The  mechanisms  used  bv  the  librarian  to  make  determina¬ 
tions  should  depend  upon  his  talents  and  ”xperience.  Each  librarv  SCOUT 
Analyst  should  develop  a  method  which  he  can  use  best  to  derive  measures 
which  are  consistent  with  effective  planning,  programming  and  budgeting 
for  services.  The  following  section,  a  method  for  generating  services 
utility  measures,  describes  the  use  of  a  few  mechanisms  which  can  fac¬ 
ilitate  weighting. 

Method  for  Generating  Services  Utility  Measures 

After  each  service  has  been  scouted  estimates  of  the  following  para¬ 
meters  should  be  recorded: 

1.  The  average  number  of  needs  met  each  week  in  each  service. 

2.  Rank  order  of  the  value  of  the  tv  'leal  need  met  in  each  service. 

For  example,  in  a  librarv  which  has  onlv  three  services,  (a) 
users  search  and  circulation,  (b)  circulation  (upon  request)* 
and  (c)  reference  search  and  circulation,  the  judgment  after 
scouting  the  services  mav  be  that  the  typical  need  met  in  users 
search  and  circulation  is  more  valuable  to  the  organization  than 
th<  tvpical  need  for  having  publications  circulated  which  were 
identified  in  accessions  bulletins,  but  less  valuable  than  the 
tvpical  need  met  In  reference  search  and  circulation.  The  rank 
order  of  the  value  of  each  service's  typical  need  are: 

Circulation  (upon  request)  is  circulation  of  titles  requested  bv  title 
and/or  call  number  when  the  user  has  not  conducted  a  catalog  search. 


reference  search  service  II 1,  users  search  service  02, 
and  circulation  service  (no  catalog  search)  0 3. 

3.  Ratings  of  the  value  of  the  typical  (average)  need  met  in  each 
service.  For  example,  assign  an  arbitrary  value  for  the  middle 
ranking  service,  e.g.,  users  search  *  10,  then  assign  value 
racings*  relative  to  10  for  the  typical  need  met  for  each  of 
the  other  services  while  observing  the  rank  order,  such  as 
circulation  (no  catalog  search)  ■  4  and  reference  search  and 
circulation  ■  12. 

With  these  data  available  we  are  able  to  derive  base  utilitv  measures 
for  each  service.  For  example,  if  there  is  an  average  of  50  needs  for  3ome 
(1  or  more)  items  (books  and  documents)  to  be  circulated  per  week  where 
there  has  been  no  previous  catalog  search  and  the  value  of  circulating  these 
items  is  rated  as  4  utils  for  the  typical  (average)  need,  the  base  utilit/ 
measure  for  this  circulation  service  is  4  X  50  ■  200  utils.  If  there  is 
an  average  of  140  users  search  needs  met  per  week  and  the  typical  need 
met  ha3  a  rating  of  10  utils,  the  base  utility  for  users  search  and  circula¬ 
tion  service  is  1400  utils.  If  there  Is  an  average  of  60  reference  search 
needs  met  per  week  and  the  rating  is  12  utixs  for  the  typical  need  met, 
the  base  utilitv  for  reference  search  and  circulation  service  is  720  utils. 

This  mechanism  can  be  used  to  derive  base  utility  measures  which 
can  be  recorded  as  the  measure  of  utilitv,  as  in  Figure  9  in  the  2nd  row 
from  the  top  (see  row  marked  6) .  Since  SCOUT  Analysis  is  intended  for 
quantifying  subjective  values,  it  is  not  necessary  that  the  base  utilitv 
measures  be  derived  by  this  method.  Furthermore,  if  this  method  is  used  for 
a  base  utility  measure,  it  is  not  necessary  to  retain  the  base  as  the 
measure  of  utilitv.  The  base  number  can  be  adjusted  according  to  values 
not  considered  in  the  development  of  the  base  utilitv.  The  base  utilitv 
car.  be  ad1u3ted  at  anv  time  during  SCOUT  Analysis  to  reflect  the  effect 
of  new  considerations.  The  utilitv  measures  si'.ould  never  be  considered 
fixed  or  static,  thev  simplv  reflect  accumulated  values  based  on  empirical 
criteria  in  the  mind  of  the  SCOUT  Analyst.  The  SCOUT  Ar.aivst  whether 
he  is  the  chief  librarian  or  another  official,  must  be  accepted  as  an 
authority  before  the  measures  can  be  given  credibility.  The  credibility 
of  value  judgments  can  be  measured  bv  the  consistency  of  judgments  between 
two  or  more  qualified  analysts.  Credibility  can  also  be  judged  according 
to  the  consistency  of  value  judgments  with  policies,  planning,  programming 
and  budgeting.  Further  credibility  checks  could  include  investigating  the 
validity  of  the  methods  used  for  generating  the  values  and  also  bv  in¬ 
vestigating  the  reliability  of  the  data  which  were  used  to  derive  the  values 
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Value  ratings  of  "typical  needs"  should  not  be  based  purelv  upon  the 
relevance  of  the  information.  A  feasible  mechanism  for  rating  the  value 
of  information  mav  be  based  upon  the  usefulness  of  the  information  in 
improving  or  reducing  the  solution  probability  of  an  approach  to  a  problem 
The  solution  probability  of  an  approach  to  a  problem  is  discussed  bv  T..i. 
Alien  in  "Organizational  Aspects  of  Information  Flow  in  Technology,'’  from 
the  Proceedings  of  the  1968  As  1 Uh  Annual  Conference.  The  anticipated 
benefits  frou  reaching  a  solution  should  also  be  considered  in  the  value 
judgments  of  Information  which  is  relevant. 


TASK  7 


Task  7  assigns  expected  changes  in  utility  according  to  changes 
in  man-hours  for  each  operation.  For  example:  if  cataloging  were 
assigned  70  man-hours  per  week  instead  of  the  existing  80  man-hours, 
we  would  expect  that  long-run  utility  of  reference  "-arch  and  circulation 
would  be  reduced.  The  amount  of  reduction  of  utility  of  this  service 
can  be  Judged  by  a  qualified  SCOUT  Analyst  according  to  empirical  criteria 
as  in  the  judgment  of  utility  of  existing  services.  The  following  section 
describes  some  mechanisms  which  may  be  used  to  facilitate  estimation  of 
utility  changes  due  to  changes  in  operation  outputs. 

Method  for  Estimating  Changes  in  Service  Utility  Due  to  Changes  in 
Operation  Outputs 

The  existing  utility  of  reference  search  and  circulation  service  was 
stimated  to  be  720  utils  (see  Figure  9).  This  figure  was  derived  from 
an  estimate  of  60  needs  met  per  week,  where  the  typical  (average)  need  was 
estimated  to  have  a  value  of  12  utils  in  relation  to  the  value  of  other 
service  needs  met.  The  reduction  of  10  man-hours /week  in  cataloging 
(from  80  man-hours /week  to  70  man-hours/week)  would  be  expected  to  have 
some  effect  on  the  number  of  needs  met  and/or  the  amount  of  relevant  in¬ 
formation  given  in  reference  search  and  circulation  service.  At  this 
point  we  can  construct  a  utility  curve.  The  existing  utility  of  the 
service  and  the  existing  output  of  the  operation  is  plotted  as  in  Figure  10 
(see  plot "Pi") .  An  estimate  is  then  made  for  the  amount  of  time  required 
for  cataloging  by  author  and  title  only.  If  this  estimate  is  30  man¬ 
hours  per  week,  an  estimate  is  made  of  the  effect  on  the  number  of  needs 
met  due  to  elimination  of  subject  indexing.  An  investigation  of  a  sample 
of  the  needs  met  may  suggest  that  only  10  of  the  60  needs  would  have  been 
met  without  subject  indexing,  and  of  these  10  needs  the  amount  of  relevant 
information  given  would  have  been  less,  and  the  value  of  the  information 
given  would  have  been  2  utils  less  relative  to  the  12  utils  of  the 
typical  need  met.  Therefore,  the  total  utility  of  the  service,  with  only 
30  standard  man-hours  of  cataloging  output  per  week,  would  be  10  x  10  or 
100.  This  utility  change  and  output  of  author  and  title  indexing  is 
plotted  in  Figure  10  (see  plot  "P2") • 

Next  a  study  can  be  made  of  a  sample  of  the  needs  met  to  determine 
the  recall  ratio.*  If  the  recall  ratio  was  .80,  an  estimate  is  made  as 
to  the  number  of  man-hours  required  for  subject  cataloging  which  would 
give  a  recall  ratio  of  perhaps  .90;  this  estimate  is  120  man-hours  per 
week  which,  we  may  judge,  would  double  the  number  of  subject  index  terms. 
i'n  estimate  is  then  made  that  the  value  of  the  typical  need  met  would 
increase  from  12  to  14  utils.  The  utility  of  the  60  needs  then  would  be 
1'  -  60  or  840.  Furthermore,  assume  that  in  the  experience  of  the 
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No.  of  relevant  documents  in  collection 


Recall  ratio 


CATALOGING  OUTPUTS  -  STANDARD  MAN-HOURS  OF  WORK 


reference  service  an  average  of  10  needs  are  not  met  during  each  week. 

The  increase  in  indexing  operations  would  be  expected  to  reduce  the 
number  of  needs  not  met,  and  therefore  increase  the  average  number  of 
needs  met.  For  example,  the  increase  in  recall  ratio  due  to  doubling 
subject  indexing  is  expected  to  satisfy  2  of  the  10  needs  which  would 
otherwise  not  have  been  met.  This  increases  the  number  of  needs  met  per 
week  to  62.  The  total  base  utility  of  reference  search  and  circulation 
service  would  then  become  62  x  14  or  866  utils.  This  point  is  plotted 
in  Figure  10  as  "P3." 

The  next  step  is  to  consider  the  effect  on  the  utility  of  reference 
search  service  if  no  catalog  is  available.  If  this  were  the  case,  a 
small  percentage  of  the  needs  may  still  be  met.  A  study  of  the  needs 
met  may  indicate  that  an  average  of  5  needs  are  met  each  week  by  the 
reference  librarian  by  recalling  information  from  memory  which  is 
relevant  and  adequate  to  satisfy  the  need.  Assume  that  the  utility  of 
this  class  of  needs  Is  estimated  as  11  utils  relative  to  the  12  utils 
typical  of  the  existing  reference  search  needs  met.  The  estimate  of  the 
base  utility  of  reference  search  and  circulation  service  with  no  catalog 
available  based  upon  11  utils  per  need  for  5  needs  met  is  11  x  5  or  55 
utils.  This  point  is  plotted  as  "P4"  in  Figure  10.  The  points  can  now 
be  joined  by  a  smooth  curve  which  represents  the  utility  curve*  of 
reference  search  and  circulation  service  with  increasing  cataloging  out¬ 
puts  (see  curve  in  Figure  10) . 

With  the  utility  curve  given,  it  is  now  possible  to  determine  the 
change  in  utility  of  the  service  with  a  given  change  in  operation  outputs. 
The  utility  change  from  80  to  70  man-hours  per  week  can  be  derived  from 
the  difference  in  utility  between  "P5"  and  "P^"  (see  Figure  10)  or 
approximately  100  utils .  The  .nan-hour  change  is  then  recorded  in  the 
total  and  marginal  utility  charge  chart  (see  Figure  9,  column  4).  The 
utility  change  from  80  to  90  man-hours  per  week  is  approximately  +55 
utils.  The  utility  change  +55  utils  is  then  recorded  in  the  schedule 
(see  Figure  9,  column  7). 

This  procedure  can  be  continued  to  fill  the  matrix  in  the  utility 
schedule  at  all  coordinates  where  operations,  materials  and  other  com¬ 
ponents  affect  service  utility. 

TASK  8 


Task  8  sums  the  utility  changes  for  each  operation  and  records  the 
sum  in  the  total  utility  change  column  of  the  utility  schedule  (see  column 
marked  8,  upper  right,  Figure  9). 


This  utility  curve  is  meaningful  only  when  all  other  components 
which  affect  utility  of  the  service  are  held  constant. 
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TASK  9 


This  task  assigns  expected  changes  in  utility  according  to  changes 
in  quantities  of  materials  such  is  books,  documents  and  periodicals  in 
the  collection.  For  example,  a  change  from  a  collection  size  of  50K 
items  to  55K  items  would  be  expected  to  give  additional  utility  to  some 
of  the  services.  Value  judgments  of  the  utility  of  a  collection  can  be 
based  on  several  criteria  such  as  the  usfer  subject  area  coverage  by  the 
collection,  the  amount  of  use  of  a  collection,  the  percentage  of  needs 
which  are  provided  with  sufficient  relevant  information  from  the  col¬ 
lection  to  satisfy  the  need.  The  criteria  used  should  be  the  choice  of 
the  SCOUT  Analyst.  The  following  section  describes  some  mechanisms 
which  may  be  used  to  facilitate  Task  9 . 

Method  for  Estimating  Changes  in  Service  Utility  Due  to  Changes  in 
Collection  Slsc 


In  analyzing  the  effectiveness  of  a  collection  we  must  think  in 
terms  of  long-run  effectiveness.  If  the  existing  collection  size  is 
50,000  and  the  percentage  of  needs  met  per  week  in  reference  search  and 
circulation  service  measures  83%  *  5%,  e.g.,  90%  of  the  time,  we  hav<-  a 
reliable  measure  of  the  effectiveness  of  the  collection  in  terms  o<'  i*  < 
probable  percentage  of  needs  met.  This  condition  (50,000  collect io.  :  *.c* 
and  83%  coverage)  can  be  plotted  on  a  utility  chart  (see  "P^,"  Figure  li). 
After  plotting  "Pi"  we  may  identify  10%,  that  is,  5000  of  the  titles  in 
the  collection.  This  can  be  accomplished  by  selecting  all  titles  with 
an  accession  number  ending  in  digit  1,  if  the  numbers  are  chronologically 
assigned  and  consecutive.  From  a  study  of  the  needs  met  we  can  determine 
the  number  of  needs  which  would  not  have  been  met  if  the  identified 
5,000  titles  were  not  in  the  collection.  Assume  a  study  shows  that  1%  ±  .5% 
of  the  needs  would  not  have  been  met  90%  of  the  time.  Then  we  can  deduce 
that  with  a  collection  size  of  45,000  titles  only  82%  of  the  needs  would 
have  been  met.  This  state  (45,000  titles  and  82%) is  then  plotted  (see 
"P2>"  Figure  11).  After  plotting  "P2"  we  shall  select  all  titles  with  an 
accession  number  ending  in  digit  2  and  plot  t’.. ;  "redi'-ted  percentage  of 
nr  ,ds  met  with  a  collection  size  of  40,000  titles.  The  study  thus  shows 
that  80%  *  5%  of  the  needs  would  have  been  met  90%  of  the  time  (see  "P3", 
Figure  11) .  The  points  are  then  joined  by  a  smooth  curve  which  represents 
the  effectiveness  of  the  collection  size  in  terms  of  needs  coverage. 

The  smooth  curve  is  extrapolated  both  forward  and  backward*  to  reflect 
an  estimated  coverage  effectiveness  curve  for  a  collection  size  of  a 
range  sufficient  for  adequate  and  confident  planning,  programming  and 
budgeting.  The  next  step  in  Task  9  is  to  determine  the  change  in  utility 
of  the  service  due  to  change  in  collection  size.  At  the  hypothetical 
library  a  reduction  of  collection  size  from  50K  to  40k  (plotted  in  Figure  9, 

*  With  sufficient  time  the  curve  for  collection  sizes  less  than  the 
existing  size  can  be  determined  without  backward  extrapolation. 
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see  column  marked  5)  will  reduce  effectiveness  of  reference  search  and 
circulation  service  by  3%  (see  Figure  11) .  This  represents  a  change  in 
utility  of  the  service  of  (-.03)  x  (720)  ■  -22  utils.  Also,  a  change  from 
50K  to  60K  (plotted  in  Figure  9,  column  5)  is  predicted  to  increase  the 
utility  by  1%.  This  represents  a  change  in  utility  of  (+.01)x(720)  ■  8  utils. 
These  utility  measures  are  then  recorded  in  thi  utility  schedule  (see  column 
marked  9,  Figure  9).  This  process  can  be  continued  until  the  utility 
schedule  is  completed  for  the  change  in  utility  of  all  services  according 
to  given  changes  in  collection  size. 

TASK  10 

This  task  sums  the  utility  of  books  and  documents  for  che  quantities 
listed  (see  column  marked  10,  lower  right,  Figure  9). 

TASK  11 


The  eleventh  task  determines  the  marginal  utility  of  the  components, 
which  is  computed  by  the  formula: 

total  utility  change 
quantity-  change 

For  operations  the  quantity  will  be  in  standard  man-hours  of  work.  For  books 
and  documents  the  quantity  will  be  the  number  of  titles.  The  marginal  utility 

of  cataioging  between  70  and  30  man-hours  output  will  be  *  -30.  The 

marginal  utility  is  then  recorded  in  the  marginal  utility  schedule  (See  Figure 
9,  columns  11  and  12).  Task  11  completes  the  schedule  of  base  utility  and 
marginal  utility  measures.  The  measures  are,  however,  subject  to  change 
according  to  criteria  not  previously  considered  in  the  analysis.  We  shall, 
therefore,  consider  other  criteria  in  Task  14  (see  p.Sfi)  and  describe  how 
the  utility  measures  may  be  altered  and  used  for  planning,  programming  and 
budgeting . 

TASK  12 

This  task  determines  if  the  collection  size  should  be  changed.  Assume 
that  at  the  present  the  library  is  acquiring  100  books  and  documents  per  week 
for  50  weeks  a  year.  This  is  a  total  of  5,000  items  a  year.  At  this  rate 
in  10  years  the  collection  size  would  increaoc  Py  50,0r9  items,  assuming 
there  is  no  weeding.  However,  most  A.T.L.'s  have  weeding  policies.  Therefore, 
let  us  assume  that  the  policy  at  the  hypothetical  library  is  to  weed  10%  of 
the  collection  each  year  In  order  to  maintain  a  collection  from  year  to  year 
which  would  be  of  equal  value  to  the  users  for  given  collection  sizes. 
Therefore,  to  maintain  a  50,000  collection  size  it  will  be  necessary  to  ac¬ 
quire  5,000  items  per  year  or  aporoximately  100  items  per  week  to  keep  the 
collection  from  depreciating  in  value.  This  expenditure  must  be  taken  from 
the  top  of  the  budget  to  make  the  utility  analysis  valid.  The  validity  of 
the  base  utility  values  for  operations  depends  upon  holding  the  value  of 
the  base  collection  size  (existing  collection  size)  constant.  For  the 
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hypothetical  library  the  base  collection  size  is  50,000  items.  We  now 
continue  the  analysis  to  determine  if  the  collection  size  should  be 
changed.  The  analysis  indicates  that  we  can  expect  approximately  3.7 
utils  per  week  increase  for  each  1,000  items  added  to  the  collection, 
which  is  in  addition  to  base  replacement  items.  A  cost  accounting  of  t’^e 
items  purchased  in  the  past  will  determine  the  cost  per  1,000  items.  This 
includes  all  costs  of  getting  the  items  in-house  and  the  costs  of  order¬ 
ing,  processing  and  cataloging,  if  additional  man-hours  arc  required  over 
the  existing  man-hours  required  for  base  replacement.  For  the-  hypothetical 
A.T.L.  let  us  assume  that  the  cost  is  $10,000  nor  1,000  items  and  that  the 
lifetime  of  the  average  title  is  10  vears.  The  amortized  cost  for  1,000 
titles  for  a  year  would  be  $1,000.  The  amortized  cost  for  a  week  would 
be  $20.00.  We  can  now  determine  if  it  is  practical  to  increase  the 
collection  size. 

Collection  size  should  be  increased,  if  MU-j/P^  of  collection  size  is 
the  largest  MUj/1’^  where  MUj  is  the  component's  marginal  utility  and  P.j 
is  the  component's  cost  per  unit.  MU*/  for  collection  size  is  MUj/Pt 
or  3.7/$20  *  .19  and  is  smaller  than  MUjf/P^;  therefore  collection  size 
should  not  be  increased  without  first  increasing  cataloging  and  other 
components  until  MU*/P^  of  all  components  is  .19  or  less.  We  should  also 
not  decrea.se  collection  size  by  weeding  in  excess  of  10%  per  year  or  by 
performing  less  than  100%  base  replacement,  since  the  decreasing  marginal 
utilitv  MUj/Pj  is  not  the  smallest  MUf/P^.  MUj/Pj  -  -9.2/5  -  -1.8. 
Circulation  hv  predetermined  listings  should  be  decreased  first,  since  its 
decreasing  marginal  utility  MU£/Pjr  ■  -3.0/S3.00  ■  -1.0  and  is  the  smallest 
MUjTVP,  for  increasing  innuts. 

1  :>e  aii!  vs  is  cl  the  marginal  utility  and  unit  cost  of  items  in  the 
collection  has  indicated  that  the  collection  base  size  should  not  be  the 
first  component  changed.  With  the  collection  size,  value  and  funding  held 
constant  we  can  now  proceed  to  analvzr  the  equilibrium  of  operations,  that 
is,  we  can  determine  if  the  funds  are  properly  distributed  between  opera¬ 
tions  to  maximize  the  utility  of  the  services  within  budget  constraints. 

TASK_i3 

This  task  determines  the  state  of  equilibrium  of  the  operations  per¬ 
formed.  the  equilibrium  condition  for  maximum  utilitv  exists  when  total 
budget  for  operations  (I);  amount  of  each  operational  output  (A,B,C, . . ,N) ; 
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price  of  each  operational  unit  (Pa,  Pb ,  Pc,...Pn)  and  marginal  utility  of 
each  operational  unit  (MUa,  MUb,  MUc...MUn)  satisfy  the  following 
equations 


(1) I«(A) (Pa)+(B) (Pb)+(C) (Pc)+. . .  +  (N) (Pn) ; 

...  MUa  MUb  MUc  Ml'n 

Pa  Pb  *  Pc  “  •**  "  Pn  * 

The  following  value  table  (Table  1)  lists  the  values  for  these  equations 
for  the  hypothetical  A.T.L.  Compare  these  data  to  data  in  columns  marked 
4,  11  and  13  of  Figure  9. 


Table  1  -  Value  Table 


I  $1240  per  week 


* 


Increasing 

MUj 


A  - 

80 

std. 

rr,  m-hrs 

of 

work 

Pa 

m 

$5.00/man-hr. 

MU& 

m 

+20.5 

B  - 

40 

std. 

man-hrs 

of 

work 

Pb 

m 

5.00/man-hr. 

mu£ 

m 

+  15.5 

C  - 

10 

std. 

man-hrs 

of 

work 

Pc 

» 

3.00/man-hr . 

Mut 

m 

+  .4 

D  - 

20 

std. 

man-hrs 

of 

work 

Pd 

m 

3.00/man~hr. 

MU& 

m 

+  1.0 

E  - 

20 

std. 

man-hrs 

of 

work 

Pe 

m 

3.00/man-hr. 

MUe 

m 

+  3.6 

F  - 

10 

std. 

man-hrs 

of 

work 

Pf 

m 

3.00/man-hr. 

Mut 

m 

+  10.0 

G  - 

10 

std. 

man-hrs 

of 

work 

Pr 

m 

5.00/man-hr. 

MUg 

m 

+  33.4 

H  - 

70 

std. 

man-hrs 

of 

work 

Ph 

m 

5.00/man-hr. 

Mut 

m 

+  15.0 

Decreasing 

MU^ 

MUa  -  -30.0 
MUb  -  -18.0 
MUc  -  -20.0 
MU3  -  -  3.0 
MUe  -  -65.0 
MUf  -  -20.0 
MUg  -  -43.6 
MUh  -  -18.0 


$1240  per  week  is  the  budget  >r  ope'..io.,u  *ist:d.  This  does  not 
include  budgets  for  overhead,  management  and  for  purchases  of  materials 
(Including  books  6  documents)  and  equipment. 


UsinR  the  values  in  Table  1,  equation  (1)  becomes 

(1)  $1240  -  (80) (5. 00) +(40) (5.00)+ ( 10) ( I. 00) 

+  (20) ( 3. 00) +(20) (3. 00) +(10) ( 3.00) 

+  (10)(5.00)+(70)(5.00) 

-  $400  +  S20o+$30+$60+$60+$ 30+S50+$350 

-  $1180. 
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From  equation  (1)  it  can  be  seen  that  the  operational  costs  do  closely 
approximate  the  budgeted  figure.  Only  $60  per  week  is  lost  in  expenses 
not  accounted  for. 


To  determine  if  the  operations  are  in  equilibrium  we  must  interpolate 
MUj ,  for  the  existing  point  on  the  total  utility  curve  for  each  operation 

MUj  -  (MUj  -  MUj)/2.  Therefore,  MUa  -  (20.5*  4-  30.0*)/2  -  50.5/2  -  25.25, 

MUb  *  16.75,  MUc  -  10.2,  MUd  -  2.00,  MUe  -  34.30,  MUf  -  15.00,  MUg  -  38.50, 

MUh  *  16.50.  The  next  step  is  to  substitute  the  values  of  MUj's  and  Pj  's  in 
equation  (2) . 

25.25  16.75  10.2  2.00  34.30  15.00  38.50  16.50 

U;:  5.00  "  5.00  *  3.00  “  3.00  "  3.00  "  3.00  "  5.00  "  5.00  5 

(2):  5.05  -  3.35  -  3.40  -  .66  -  11.43  -  5.00  -  7.70  -  3.30. 


Equation  (2)  is  not  in  equilibrium.  Therefore,  the  operations  are  out 
of  equilibrium  and  funds  should  be  budgeted  to  improve  this.  Funds  should 
be  added  to  operations  where  MUj/Pj  is  largest  and  funds  should  be  removed 


from  operations  where  MU^/P^ 


is  smallest. 


TASK  14 


The  fourteenth  task  determines  which  operations  should  be  reduced  first 
and  which  should  be  Increased  first  if  they  are  not  in  equilibrium.  The 

M«j"l  values  (see  positive  values  in  column  marked  14,  Figure  9)  from 

highest  to  lowest  are:  user  education  actions  4-6.68,  cataloging  action  4-4.10, 
accessions  bulletin  preparation  and  distribution  4-3.33,  performance  of 
searches  4-3.10,  acquisitions  actions+3.0,  preparation  and  maintenance  of 
collection  4-1,2,  circulation  by  predetermined  listings  actions  4-, 33, 
circulation  upon  request  actions  4-. 13.  This  is  the  order  in  which  funds 
should  be  added,  first  to  user  education,  second  to  cataloging  actions, 
and  so  on.  If  no  additional  funds  are  allocated,  it  will  be  necessary  to 
reduce  funds  from  some  operations  where  it  is  possible  to  approach  equilibrium. 
Funds  should  be  reduced  where  MUj/Pj  is  smallest  (see  negative  values 

in  column  narked  14,  figure  9).  The  first  reduction  should  he  in  circula¬ 
tion  bv  nrcdctermincd  listings  where  Mtfp/Pg  ■  -1.00,  the  2nd  reduction 
should  be  in  performance  of  searches  where  MUjj/Pg  »-3.60,  and  so  on. 


See  column  marked  11,  in  Figure  9 


50 


Before  predetermined  circulation  is  reduced  to  increase  user  education 
actions,  we  must  consider  the  short-run  and  long-run  effects.  Many  factors 
should  be  considered  before  such  a  change  is  made,  such  as  (1)  Can  pre¬ 
determined  circulation  be  made  more  efficient  to  reduce  the  cost  per  item 
circulated  sufficiently  to  increase  MUg/P^  above  the  level  of  the  second 

lowest  MUj/Pj  operation?  (2)  Would  the  long-run  advantages  of  increasing 

user  education  offset  the  short-run  disadvantage  of  "ocr  public  relations 
effects  of  reducing  predetermined  circulation  services  which  the  users 
have  become  accustomed  to  receiving?  (3)  Would  a  sufficient  number  of  users 
accept  reduced  predetermined  circulation  and  increased  user  education  to 
the  extent  that  MUq/Pq  can  be  increased  above  the  level  of  the  second 
lowest  MUT/p4?  (4)  Would  officials  in  the  organization  accept  the  proposals 
for  increased  user  education?  The  number  of  relevant  factors  to  be  con¬ 
sidered  before  making  such  a  change  would  in  most  cases  exceed  4,  but  many 
of  them  would  not  be  high  priority  considerations. 

The  next  step  is  to  determine  if  anv  factors  are  effective  barriers 
to  reduction  of  predetermined  circulation  or  to  increasing  user  education. 

For  example,  if  none  of  the  proposals  for  user  education  programs  are 
accepted  regardless  of  the  effort  - ade  to  develop  the  programs,  factor  4 
may  be  an  effective  barrier  to  utility  maximization.  Again,  officials  mav 
have  convictions  that  the  value  of  user  education  is  overestimated.  If 
this  is  the  case,  the  SCOUT  Analyst  should  revise  the  estimates  of  utilitv 
according  to  valid  utility  considerations  introduced. 


After  thorough  consideration  of  the  factors  relevant  t>  making  the 
change  and  if  factor  MUp/Pp  remains  the  lowest  MU’j'/Pj  after  changes  in 

utilitv  estimates  and/or  efficiency  and  if  there  are  no  effective  barriers 
outstanding,  a  reduction  should  be  planned,  proposed  to  top  management  and 


implemented  if  approved. 


If  'Tu'^/Pp  continues  to  be  the 


lowest  NU^/P^  and 


if  there  are  effective  barriers  such  as  (1)  nonutilitarian  considerations 
or  (2)  factors  which  would  prohibit  released  labor  from  assuming  more 
utllitv-maximizlng  tasks,  the  reduction  should  not  necessarily  be  made. 


Increases  in  operational  activity  should  he  implemented  for  operations 
where  Ml/j/P^  remains  the  highest  after  utilitv  and/or  efficiency  adlust- 
ments  are  made,  unless  there  are  effective  barriers 


Where  barriers  exist,  reductions  or  increases  should  be  made  on  opera¬ 
tions  to  maximize  utilitv  until  the  barriers  become  effective;  that  is, 
until  nonutilitarian  considerations,  considerations  of  costs  not  paid  bv 
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the  library  (as  user  time)  and  other  extenuating  circumstances  prohibit 
further  reduction  of  low  MUj/P.,  operations  or  further  increase  of  high 
MUj/pj  operations. 

After  completion  of  Tash  14  the  utility  schedule  should  be  complete. 
That  is,  all  appropriate  adjustments  of  utility  should  have  been  made, 
and  meaningful,  useful  effectiveness  measures  are  available  as  a  basis 
for  planning,  programming  and  budgeting. 


i 

1 

I 


VI.  CORE  ANALYSIS— DETAILED  DESCRIPTION* 


Tr'.a  usage  of  CORE  Analysis  will  be  described  for  cataloging;  however, 
it  will  h®  useful  iti  all  areas  where  operational  costs  and  quantity  cor¬ 
relate  highly  among  A.T.L.’s  and  where  effectiveness  (quality  of  outputs) 
of  the  operations  at  each  library  can  be  placed  in  definable  quality 
classes . 

TASK  l 

The  first  task  is  to  plot  the  cost  and  output  of  the  operation  at 
each  library  participating  in  the  analysis.  (See  Figure  12). 

TASK  2 


The  second  task  computes  the  coefficient  of  correlation  (r). 


Al  \-  \  I'i  i  v-v  )“ 


The  values  and  their  meaning  are  found  in  Table  2  for  a  hypothetical  sample 
ca&e.  The  cost  vs.  the  number  of  titles  cataloged  in  this  example  has  a 
correlation  coefficient  of  approximately  .57.  The  highest  possible  cor¬ 
relation  is  +  1.00,  The  correlation  coefficient  indicates  that  a  unit 
cost  standard  for  all  cataloging  would  be  too  loose.  We  should,  therefore, 
attempt  to  identify  classes  of  cataloging  in  terms  of  quality. 

TASK  3 


If  the  correlation  is  weak,  the  next  task  is  to  develop  a  quality 
continuum  for  cataloging  (see  Figure  13)  and  identify  the  position  of  each 
library's  cataloging  quality  by  sampling. 

In  this  hypothetical  example^all  libraries  which  do  only  descriptive 
cataloging  will  be  placed  in  class  1;  let  us  assume  these  libraries  are 
Nos.  8,  23,  19,  44,  13,  38,  48,  28,  39  and  47. 

TASK  4 

This  task  computes  the  coefficient  of  correlation  (r)  for  the  10 
libraries  doing  only  descriptive  cataloging  The  values  in  Table  2  are 
substituted  in  the  equation  and  the  value  of  r  is  determined  as  follows: 


A 

CORE  Analysis  is  discussed  in  principle  in  Chapter  VI  of  the  Phase  II 
report,  p.  49  through  38  under  data  correlation  considerations. 

**  The  correlation  coefficient  r  may  be  used  as  an  index  measuring  the 
closeness  of  fit  of  the  points  to  a  least  squares  regression  line  of 
best  fit  (see  Task  5) . 
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-OUTPUT  (NUMBER  OF  TITLES  CAT  ALOGEO 


TABLE  2 


Point 

X 

Y 

(x  -3?) 

1 

200 

300 

29.88 

2 

40 

100 

-130.12 

3 

180 

500 

*  9.88 

4 

70 

400 

-100.12 

5 

70 

150 

-iOO.12 

6 

300 

440 

129.88 

7 

120 

300 

-  50.12 

8 

20 

10 

-150.12 

9 

300 

770 

129.88 

10 

220 

500 

49.88 

11 

230 

380 

59.88 

12 

130 

190 

-  40.12 

13 

100 

100 

-  70.12 

14 

286 

350 

115.88 

15 

240 

300 

69.88 

16 

260 

400 

-  89. *8 

17 

140 

390 

-  30.12 

18 

100 

250 

-  70.12 

19 

80 

100 

-  90.12 

20 

60 

100 

-110.12 

21 

60 

300 

-110.12 

22 

40 

200 

-130.12 

23 

40 

50 

-130.12 

24 

260 

700 

89,88 

25 

160 

250 

-  10.12 

26 

280 

500 

109.88 

27 

180 

400 

9.88 

28 

190 

160 

19.88 

29 

700 

700 

29.88 

30 

166 

770 

-  4.12 

31 

260 

900 

89.88 

32 

220 

800 

49.88 

33 

160 

500 

-  10.12 

34 

160 

200 

-  10.12 

35 

180 

300 

9.88 

36 

200 

200 

29.88 

37 

260 

490 

89.88 

36 

120 

100 

-  50.12 

39 

212 

270 

41.88 

40 

246 

450 

75.88 

41 

200 

400 

29.88 

42 

140 

280 

-  JO.  12 

43 

104 

150 

-  66.12 

44 

90 

60 

-  80.12 

45 

280 

450 

109,88 

46 

254 

290 

83.80 

47 

228 

240 

57.88 

48 

142 

130 

-  28.12 

49 

146 

810 

-  24.12 

50 

182 

950 

u.es 

N”total  no.  of 

Z  X  ”sum  of 

*  Y  =sum  of 

points  *  50 

column  X  =  8506 

column  Y=  1 8030 

X  =  average  of  no« .  Y  =  overage  of  no, . 
in  column  X  in  column  Y 

“  *  X/N  -  3506/50  =  I  Y/N  =  1 8030/50 
-  >70.1  a  ”360.60 

r(X-X)  (y-7) 
r  m  V^(X  -  X)2  (Y  -  7)2 
m  LUL*  iq5 

Bi.Ox  10 10 


(Y  -  Y)2  f 

3672.4 

67912.4 

19432.4 

1552.4 

44352.4 

6304.4 

3672.4 

122920.4 

167608.4 

19432.4 

376.4 

29104.4 

67912.4 

112.4 

3672.4 

1552.4 

864.4 

12232.4 

67912.4 

67912.4 

3672.4 

25792.4 

96479.4 

115192.4 

12232.4 

19432.4 

1552.4 

40240.4 

115192.4 

167608.4 

290952.4 

193072.4 

19432.4 

25792.4 

3672.4 

25792.4 

16744.4 

67912.4 

8028.4 

7992.4 

1552.4 

6496.4 

44352.4 

90360.4 

7992.4 

4984.4 

14544.4 

53176.4 

201960.4 

347392.4 

I  (Y  -  7)2  -  jum  of 
of  nos,  in  column 
(Y-Y)2  =  2738292.0 


5. 14  x  I05 


V  (2.96  x10s)  (27.4  x  I05) 
5. 14  x  105 
9. Ox  10s 


(X  -  X)2 

(Y-Y) 

892,8 

-  60. 6 

16931.2 

-260.60 

97.6 

139.40 

i0024.0 

39.40 

10024.0 

-210.60 

16868.8 

79.40 

2512.0 

-  60.60 

22536.0 

-350.60 

16868.8 

409.40 

2488.0 

139.40 

3585.6 

19.40 

1609.6 

-17Q.60 

4916.8 

-260.60 

13428.2 

-  10.60 

4883.2 

-  60.60 

8078.4 

39. xn 

907.2 

29.40 

4916.8 

-110.60 

8121.6 

-260.60 

12126.4 

-260.60 

12126.4 

-  60.60 

1693’ .2 

-160.60 

16931.2 

-310.60 

8078.4 

339.40 

102.4 

-110.60 

12073.6 

139.40 

97.6 

39.40 

395.2 

-200.60 

892.8 

339.40 

17.0 

409.40 

8078.4 

539.40 

2488.0 

439.40 

102.4 

139,40 

102.4 

-160.60 

97.6 

-  60.60 

072.8 

-160.60 

8078.4 

129,40 

2512.0 

-260.60 

1755.9 

-  90.60 

5757.8 

89.40 

892.8 

39.40 

907.2 

-  80.60 

4371.9 

-210.60 

6419.2 

-300.60 

1 2073.6 

89.40 

7035.9 

-  70.60 

3350.1 

-120.60 

790.7 

-230.60 

581.6 

449.40 

141.1 

589.40 

I(X  -  5<)2=  sum  of 
no, .  In  column 
(X-X)2  =  295890.8 


X  -  X)  <Y  -  Y1 

-  1810.7 

33909.3 
*  1377.3 

-  3944.7 

21085.3 

10312.5 

3037.3 

52632.1 

53172.8 

6953.3 
1161.7 

6844.6 

18273.3 

-  1228.3 

-  4234.7 

3541 .3 

-  885.5 

7755.3 

23485.3 

28697.3 

6673.3 

20897.3 

40415.3 

30505.3 

1119.3 

15317.3 

389.3 

-  3987.9 

10141.3 

-  1686.7 

48481.3 

21917.3 

-  1410.7 

1625.3 

-  598.7 

-  4798.7 

11630.5 

13061.3 

-  3794.3 

6783.7 

1177.3 

2427.7 

13924.9 

24084.1 

9823.3 

-  5921.9 

-  6980.3 

6484.5 
-10839.5 

7002.1 

I(X-X)  ■ 
(Y-Y)  ~  sum 
of  the  products 
no,,  in  columns 
(X-K)  ana 
(Y-Y)  - 
513997.7 
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.95 


49.3  x  10 3 
¥27.2  x  108 


49.3  x  103 _ 

(5.22  x  10^) (60.4  x  103) 


The  cost  vs.  the  number  of  titles  cataloged  descriptively  for  class  1 
has  a  correlation  coefficient  of  .95.  Any  correlation  above  .90  should 
be  considered  adequate  for  developing  a  tight  standard  for  the  class  in 
the  quality  continuum.  If  the  correlation  coefficient  is  below  .80  for  a 
class,  the  class  should  be  divided  further  on  the  quality  continuum.  This 
dividing  process  should  continue  until  a  correlation  of  .80  or  better  is 
reached  or  until  no  further  distinction  in  quality  can  be  made. 

For  quality  classes  which  cannot  be  divided  further  or  which  have  two 
or  less  members  or  which  have  a  correlation  coefficient  lower  than  .80, 
it  is  recommended  that  GAME  Analysis  be  used  Instead  of  CORE  Analysis  for 
developing  standards, 

TASK  5 


This  task  computes  the  line  of  best  fit  for  the  points  in  a  quality 
class  by  the  method  of  least  squares.  The  equation  for  the  line  of  best 
fit  can  be  obtained  by  solving  the  following  simultaneous  general 
equations  (1)  and  (2)  for  a  and  b  : 

(1)  y  *  n  4  1>/ x  where:  n  ■  number  of  points  in  the  class 

(2)  •'  x  y  »  a>  x  +  I-'  x  J 

and  by  substituting  the  values  of  a  and  b  in  the  following  general  equation 
(3)  for  the  line  of  best  fit  • 

(3)  y  »  a  +  bx  . 

Using  the  values  in  Table  3,  equations  (1)  and  (2)  become 

(1)  1220  *  a(10)  +  b  (1222) 

(2)  198420  -  *(1222)  +  b(194092) 
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In  solving  the  simultaneous  equations ^first  one  constant  (a  or  b) 
is  eliminated  as  follows: 


multiply  (1)  by  -1,222  and  name  the  new  equation  (4) 

(4)  -1,490,840  -  (-12, 220) a  +  (-1 ,493,284)b  ; 
multiply  (2)  by  +10  and  name  the  new  equation  (5) 

(5)  1,984,200  =  (12 ,220)a  +  (1 ,940,920)b; 
add  equation  (4)  and  (5)  and  solve  for  b: 


(4)  -1,490,840  -  -12,220a  -1,493,284b 

(5)  1,984,200  -  12,220a  +  1,940,920b 

493360  -  447636b 


b 


493360 

447636 


1-15 


substitute  b  in  equation  (1)  and  solve  for  a  : 


(1)  1220  -  10a  +  1344.2 

10a  -  -124.2 
a  -  -12.4; 


substitute  a  and  b  in  equation  (3)  and  name  it  equation  (6) 

(6)  v  -  -12.4  +  l.lx 


where  y  ■  weekly  cost  of  descriptive  cataloging, 
and 


x  ■  the  number  of  titles  descriptively 
cataloged  per  w^k. 


TASK  6 


Task  6  is  to  draw  the  line  of  best  fit  in  the  scatter  diagram 
(see  Figure  12).  This  is  accomplished  by  connecting  any  two  points  which 
satisfy  equation  (6),  such  as  the  two  points  described  below  ; 

point  (1)  when:  x  *  100,  y  ■  110-12.4  »  97.6; 

point  (2)  when:  x  «  300,  y  -  330-12.4  -  317.6. 
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The  line  y  =  -12.4  +  l.lx  is  the  standard  for  descriptive  cataloging 
where  the  cost  (y)  should  be  $1.10  times  the  number  of  titles  cataloged 
descriptively  (x)  less  $12.40. 

CORE  Analysis  and  Performance  Control 

Libraries  which  deviate  most  from  the  line  of  best  fit  on  the  costly 
side  should  be  investigated  to  determine  if  the  higher  costs  are  justifiable. 
Library  # 39  should  be  compared  with  a  library  working  closer  to  the  standard 
which  performs  approximately  the  same  volume  of  work,  such  as  library  #47. 

If  the  comparison  shows  that  library  #39  does  not  have  more  effective 
descriptive  cataloging  and  if  extenuating  circumstances  are  not  apparent, 
library  #39  should  be  expected  to  improve  methods  and  efficiency  and  to 
work  to  the  standard. 

Libraries  which  deviate  most  on  the  low  cost  side  of  the  line  of  best 
fit  should  be  investigated  to  determine  if  the  quality  of  cataloging  should 
be  higher. 

After  an  analysis  of  the  deviates  by  comparison  with  the  conformists, 
sufficient  experience  should  have  been  gained  to  declare  the  standard  and 
to  formulate  policies  for  performance  control  by  standardization  of  costs 
and  quality. 

The  development  of  CORE  standards  is  independent  of  methods  used  in 
processing.  CORE  standards  should  be  used  to  control  input  costs  and 
output  quality.  The  methods  used  in  processing  and  the  individual  operator's 
efficiency  are  not  studied  bv  the  CORE  Analyst.  CORE  standards  can  be 
developed  without  imposition  of  standard  times  or  standard  methods  upon  the 
operations.  As  long  as  costs  and  quality  do  not  deviate  bevond  the  desired 
normal  ranges,  the  operations  managers  are  free  to  select  the  methods  to 
be  used  for  operations.  Where  CORE  Analysis  fails  to  provide  a  tight 
standard,  it  will  be  necessarv  to  define  the  methods  to  be  used  and  develop 
standards  for  elements  of  work.  Chapter  VII  discusses  GAME  Analysis  — 
a  technique  for  developing  standard  methods  and  standard  times  or  costs 
for  accomplishing  a  desired  output. 

CORE  standards  can  be  used  to  measure  the  relative  efficiency  of 
operations  between  A.T.L.'s  for  given  levels  of  effectiveness.  The  index 
of  efficiency  (E)  is  computed  bv 

a  +  bx 

1  *  "*c —  ' 

where  C  is  the  actual  cost  for  production  of  x.  If  E  is  below  1.0,  the 
operational  efficiency  is  below  par.  If  E  is  above  1.0,  the  operational 
efficiency  is  above  par. 
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VII.  GAME  ANALYSIS*— DETAILED  DESCRIPTION 


TASK  1 


The  first  task  in  GAME  Analysis  is  to  construct  a  flow  cha-t  of  the 
activities  in  a  process.  For  example,  Figure  14  is  a  flow  chart  of  bock 
cataloging  actions,  not  including  production  of  catalog  cards. 

In  flow  charting,  each  existing  activity  which  accrues  significant 
costs  should  be  identified  and  placed  in  proper  flow  sequence. 

Some  criteria  for  choosing  activities  are: 

(1)  Each  activity  should  be  performed  completely  by  one  operator 
for  a  given  title.  When  two  or  more  men  perform  one  operation, 
the  activities  of  each  man  should  be  branched. 

(2)  The  activities  should  be  broken  down  and  defined  in  adequate 
detail  to  determine  exactly  which  work  is  performed.  All 
signifies  it  work  performed  in  the  process  should  be  in¬ 
corporated  in  some  activity. 

(3)  Each  activity  should  be  performed  several  (two  or  more)  times 
before  the  next  activity  is  executed. ^  For  example,  several 
titles  are  searched  for  a  precedent,  then  several  tides  are 
placed  in  either  a  precedent  pile  or  an  original  cataloging 
pile, then  several  titles  are  cataloged  originally,  and  so  on. 

TASK  2 

This  task  determines  the  average  time  per  work  unit  for  each  activity 
and  the  reliability  of  the  time  values.  The  first  step  in  this  task  is 
to  brief  each  operator  in  the  particular  work  area  on  the  purpose  of  and 
their  roles  in  GAME  Analysis.  Each  operator  will  receive  Form  2  and  a 
list  of  instructions  on  an  operator's  data  sheet  guide.  The  data  sheet 
guide  will  explain: 

(1)  The  flow  chart  (see  Figure  14). 

(a)  The  activity  breakdown. 

(b)  The  activity  symbols  (such  as  for  in-house  shelf 
list  search). 

(c)  The  begin  time  for  an  activity. 

(d)  The  end  time  for  an  activity. 

*  GAME  Analysis  is  discussed  briefly  in  Chapter  III  of  this  report. 

**  A  cataloging  precedent  is  any  recorded  cataloging  data  on  a  given 
title  which  have  been  accepted  by  authority  is  being  valid  cataloging 
information.  Library  of  Congress  proof  slips  and  the  National  Union 
Catalog  listings  are  potential  cataloging  precedents. 
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(2)  The  kind  and  amount  of  work  done. 

(a)  The  kind  of  work  for  which  the  activity  was  performed 
between  the  begin  time  and  the  end  time  for  the  activity. 

For  example,  if  operation  G2  is  being  performed,  the 
kind  of  work  unit  would  be  original  cataloging  and  re¬ 
cording  of  catalog  data  on  the  process  sheet. 

(b)  The  amount  of  work  completed  between  the  begin  and  end 
time  for  the  activity.  For  example,  if  activity  G2  is 
performed  completely  for  2  titles  between  10:00  AM  and 
11:00  AM,  the  number  of  work  units  done  would  be  2  titles. 

(3)  The  time  spent  on  charted  activities  should  be  the  only  time  for 
which  the  operator  must  account.  For  example,  personal  time, 
make-work  activities  and  other  activities  not  directly  related 
to  performance  of  the  charted  activities  will  not  be  accounted 
for  bv  the  operator. 

The  second  step  is  to  conduct  a  work  sampling  study  in  order  to  collect 
data  to  determine  standard  times  for  each  activity.  Form  2  is  completed  by 
each  operator  in  a  given  work  area  (see  Exhibit  1,  p.  63). 

While  the  operators  are  performing  the  activities,  a  work  sampling 
studv  is  conducted  (see  Exhibit  2).  The  GAME  Analysis  work  sampling  study 
form  is  used  for  recording  data  concerning: 

(1)  The  amount  of  personal  time  spent,  symbol  P. 

(2)  The  amount  of  time  spent  in  transporting  materials  out  of 

or  into  the  work  area,  symbol  T. 

(3)  The  amount  of  time  spent  in  receiving  or  Riving  instructions, 
symbol  I. 

(U)  The  amount  of  time  spent  in  unavoidable  delays,  symbol  U. 

(5)  The  amount  of  time  spent  related  to  GAME  Analysis,  symbol  G. 

(6)  The  amount  of  time  spent  working  on  activities,  symbol  X. 

(7)  The  average  pace  of  working  of  each  operator  on  each  activity, 
see  pace  rows. 

(h)  i.xp lanat ions  of  elements--that  is,  anv  notes  to  facilitate 

understanding  of  the  elements  of  work  or  nonwork,  see  footnote 
numbers  in  footnote  rows  and  notes  at  bottom  of  Exhibit  2. 
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GAME  Analysis 
OPERATOR'S  DATA  SHEET 


Sheet 


1 


1 


OPERATOR 


Mrs.  Smith 


ACTIVITY 
fuse  symbol  -  - 
see  operator's 
data  sheet  guide' 


TIME  TIME 
BEGIN  END 


DATE  Oct.  2,  1968 
WORK  UNITS  PRODUCED 

/P.nd  and  amount  of  wor*.  done  • 
see  operator's  data  sheet  guide) 


Original  cataloging  and 


recording  catalog  data  on 


process  sheet  -  2  titles . 


Revision  of  original  cataloging 


1 1 :45  -  5  titles . 


Original  cataloging  and 


recording  catalog  data 


on  process  sheet  - 


4  titles . 


Partial  original  cataloging 


and  recording  cotolog 


data  on  proce<s  sheet  - 


I  title. 


Rev  ision  of  original 


5:00  cataloging  -  18  titles. 


Exhibit 

1 
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Exhibit  2  -  sheet  1  of  2 


GAME  Analysis 
»Voi  l  Sampling  Stud  .  Fct 


Exhibit  2  —  sheet  2  of 


An  element  in  recorded  for  each  operator  each  minute  during  the 
study.  Each  operator  is  observed  once  at  random  intervals  during  each 
minute.  The  element  of  work  or  nonwork  being  performed  at  the  instant 
of  first  observation  will  be  recorded  as  the  element  for  each  respective 
minute . 

The  third  step  summarizes  the  work  sampling  study  data  (see  Exhibit  3). 
Activity  G 2  taken  from  Exhibit  1  beginning  at  10:00  AM  and  ending  at  11:00 
AM  took  a  total  time  of  60  minutes.  However,  the  work  sampling  study  form 
(see  Exhibit  2)  must  be  investigated  before  the  activity  time  is  recorded. 
Exhibit  2j sheet  i  of  2 ^ shows  that  Mrs.  Smith  spent  2  minutes  on  personal 
time  and  2  minutes  in  instruction  time  between  10:00  and  11:00:  this  is 
a  total  of  4  minutes  not  on  the  activity,  and  therefore  the  time  on  activity 
G2  is  60-4  *  56  minutes. 

The  average  pace  rating*  is: 

[  100%+100%+95%+100%+100%+10C%-»-llQ%+105%+lQO%+10Q%+100%d-100%] 

...  . ~  12 

or 

~~~  *=  100.9% 


Personal  time  i|  2  minutes.  Instruction  time  is  2  minutes.  Leveled  work 
time  is  100.9%  of  56  minutes  =  56.5  minutes.  The  number  of  work  units 
produced  is  2.  All  activities  which  are  totally  performed  for  a  given  number 
of  work  units  are  summarized  as  in  Exhibit  3.  Partial  performance  of  an 
activity  such  as  was  performed  by  Mrs.  Smith  between  2:36  and  3:00  (see 
Exhibit  1)  will  not  be  summarized. 

The  fourth  step  determines  the  average  time  for  each  activity.  To 
obtain  a  single  estimate  of  the  time  for  performing  an  activity  per  title, 
the  leveled  work  time  is  divided  by  the  number  of  work  units  produced.  In 
Exhibit  3,  three  single  estimates  of  the  time  required  to  perform  activity 


G2  are  available. 

These 

are  ; 

(1) 

56.5  _ 

O 

t. 

28.3 

min. / title , 

and 

(2) 

120 

4 

30.0 

min. /title , 

and 

(3) 

160 

—  as 

5 

32.0 

min. /title . 

For  discussion  of  pace  rating  see  Nadler,  Gerald.  "Motion  and  Time 
Study."  New  York:  McGraw  Hill,  1955,  Chapter  23. 

Pace  ratings  are  not  essential  to  developing  good  standards.  Pace 
can  be  assumed  to  be  100%  for  the  group,  if  pace  leveling  cannot 
be  done  with  confidence. 
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Exhi 


GAME  Analysis  Sheet 

Oct.  2,  1968  Work  Sampling  Summary  Sheet 


Since  the  three  single  estimates  vary,  it  will  be  necessary  to  derive 
a  weighted  estimate  for  the  average  time  per  title. 

Single  estimate  //I  is  recorded  twice,  as  in  Table  4;  single  estimate 
It 2  is  recorded  4  times  and  single  estimate  #3  is  recorded  5  times.  Hie 
number  of  times  the  single  estimate  is  recorded  depends  upon  the  number 
of  work  units  produced.  Each  of  the  individual  recordings  will  be  called 
a  sample  member.  The  sample  member  single  estimates  will  be  designated  as 
yj,  where  i  *  l,2,3,...n,  and  yj_,  y2«  y-j,...yn  are  sample  members  It  1,  111, 

//3,...//n,  respectively. 

The  average  time  for  an  activity  is  £  yi/n.  The  sample  members 
estimate  that  the  average  time  for  activity  G2  is  336.6/11  or  30.6  min. /title. 


It  is  desirable  that  a  minimum  level  of  confidence  or  reliability  be 
attained  for  the  estimates  of  average  time  per  work  unit  of  each  activity. 
The  5th  step,  therefore,  will  be  to  determine  the  number  of  sample  members 
required  to  meet  the  minimum  level  of  confidence.  Wien  the  confidence 
level  is  ±  10%  y,  95%  of  the  time,  the  following  formula  is  used  to  compute 
N'  (the  number  of  sample  members  required  to  me^t  the  desired  confidence 
level)  .* 


N' 


20(R) (n) 
<*2  - 


2 


where:  R  is  the  range  of  vj  values. 


n  is  the  number  of  sample  members. 


d2  is  a  constant  for  a  given  sample  size 
(see  Table  4  for  d2  values) . 


For  operation  G2  the  N'  required  to  give  the  desired  confidence  in 
v  is  j 


(2  ) (32-28, 3)  (11) 
(3.173) ( 336.6) 


814 
10  70 


2 


( -76) 2 


.578 


Since  N'  is  smaller  than  n  (n»ll)  ,  no  additional  sample  members 
are  required.  If  N'  is  larger  than  n,  then  N'-n  additional  members  must 
be  gained  bv  further  work  sampling.  After  N'  total  sample  members  are 
available,  the  N'  tests  and  further  samplings  are  repeated  until  N'  is 
equal  to  or  smaller  than  n.  Bv  this  process  an  average  time  per  title 
is  eventually  derived  with  the  required  confidence.  The  standard  time  is 
then  recorded  on  or  to  the  left  of  the  line  entering  the  respective 
activity  (see  Figure  14). 
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Single  Estimates 


Sample  member 

yi 

n 

d2 

#  1 

28.3 

If  2 

28.3 

2 

1.128 

#  3 

30.0 

3 

1.693 

#  4 

30.0 

4 

2.059 

#  5 

30.0 

5 

2.326 

#  6 

30.0 

6 

2.534 

#  7 

32.0 

7 

2.704 

#  8 

32.0 

8 

2.847 

if  9 

32.0 

9 

2.970 

If  10 

32.0 

10 

3.078 

If  11 

32.0 

11 

3.173 

— 

12 

3.258 

13 

3.336 

-Vi 

-  336.6 

14 

3.407 

y 

n  s  yj 

15 

3.472 

n 

16 

3.532 

y 

„  336.6 

11 

17 

3.588 

18 

3.640 

V 

-  30.6 

19 

3.b89 

20 

3.735 

21 

3.778 

22 

3.819 

23 

3.858 

24 

3.895 

25 

3.931 

30 

4.086 

35 

4.213 

40 

4.322 

45 

4.415 

50 

4.498 

Table  4 
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TASK  3 


Task  3  determines  the  utilization  of  the  group.  Our  example 
describes  the  steps  in  computing  the  index  of  utilization  of  the  catalog¬ 
ing  process  represented  in  Figure  14.  The  1st  step  requires  an  account¬ 
ing  of  the  books  processed  in  a  given  period  of  time,  e.g.,  1  week,  to 
determine  the  percentage  of  total  titles  processed  through  each  activity 
during  that  week.  These  percentages  are  then  recorded,  (Figure  14)  below 
or  to  the  right  of  the  line  entering  the  activity. 

The  2nd  step  computes  the  "Should-take"  time  for  the  titles  processed 
during  the  week.  This  is  accomplished  by  solving  the  following  equation: 

Should-take  time  -  [ (ti) (f3)+(t2) (f2)+(t3> (f3>+. . . (tn) (fn) ]N 
where: 

ti  -  the  first  activity  time, 
i.e. ,  t3  -  .1 

t2  *  the  2nd  activity  time, 
i.e.,  for  activity  B2, 

t2  -  1.5 

t3  -  1.4 


-  .2 


tn  ■  (4.0  x  .30)+(5.0  x  . 70 ) —4.7 
and 

N  »  the  total  number  of  titles  catalogc  i  for  the  week. 


fl  -  the  fraction  of  titles 
processed  through  first 
activity,  i.e., 

fl  -  1.00 

f2  ■  the  fraction  of  titles 

processed  through  the  2nd 
activity,  i.e., 

f2  -  1.00 

f3  -  .70 

f4  -  .30 


fn  -  .30  +  .70  -  1.00 
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For  the  example,  in  Figure  14  Should-take  time 

»  [ ( .1) (1 .00)+(l .5) (1 .00) +(1.4) ( . 70)+ ( . 2) ( . 30) 

+(5.5) ( . 35)+( . 1) ( . 20) +( .05) ( . 20)+(30. 6) ( . 20) 

+(6 .0) ( .20) +(14 .0) ( . 70)  +  ( . 2)  C30)+( . 7) (.30) 

+( .05) ( . 30)+(l .5)(.30)+(1.4)(.30)+(10.0)(. 30) 
+(.4)(.35)+(.2)(.15)+(.05)(.50)+(13.5)(. 50) 

+(1 . 5) ( . 50)+ (4 . 10) ( . 50)+(4 .0  x  .30)+(5.0  x  70)]N 
«  [.10  +  1.50  +  .98  +  .06  +  1.93  +  .02  +  .01  +  6.12 

+  1.20  +  9.80  +  .06  +.21  +  .02  +  .45  +  .42  +  3.00 

+  .14  +  .03  +  .03  +  6.75  +  .75  +  2.05  +  1.20  +  3.501N 

^  40  N 

If  the  number  of  titles  cataloged  in  the  week  is  200,  the  total 
Should-take  time  is  40.00  x  200  -  8000  min.  This  is  approximately  133.3 
hours  for  the  week. 


The  3rd  step  determines  the  utilization  of  the  group.  The  measure 
of  utilization  to  be  used  1  ere  is  the  index  o.'  productivity  (I),  where: 

I  a  -5.hoiibL“JA‘'d'-  1  _ _  Standard  earned  hours  . 

total  time  charged  to  the  activity  measured  hours 


If  for  the  week  discussed  in  steps  1  and  2  the  total  man-hours  charged 
to  cataloging  is  190  man-hours. 


I 


133.3 

190 


.702  . 


TASK  4 

Task  4  is  a  methods  study.  Two  key  questions  should  be  answered  in  this 
study : 

(i)  Should  the  present  method  be  performed  more  efficiently? 

(a)  Are  personal  time,  transport  time,  Instruction  time, 

unavoidable  delay  time  or  other  nonwoik  element  times  in 
excess  of  allowances  due  to  poor  management  of  flow, 
personnel  or  training? 


See  methods  study  XI  in  Phase  II  report.  Appendix  A. 


71 


(b)  Is  the  average  pace  for  the  group  below  the  expected 
pace  and  not  due  to  factors  intrinsic  in  the  method? 


(2)  Should  the  present  method  be  changed  to  improve  efficiency? 

(a)  Are  personal  time,  transport  time,  instruction  time, 
unavoidable  delay  time  or  other  nonwork  element  times 
in  excess  of  allowances  and  due  to  factors  intrinsic 
in  tne  method  or  can  a  more  efficient  method  be 
Implemented? 

The  following  steps  answer  the  two  key  questions  for  a  sample  case 
and  describe  methods  analysis  of  the  example  method  of  cataloging  charted 
in  Figure  14.  The  1st  step  is  to  determine  if  the  present  method  could 
be  performed  more  efficiently  by  improving  management  of  flow,  personnel 
or  training.  Since  the  index  of  productivity  is  low  (.702)*,  we  should 
answer  question  1(a)  by  investigating  time  spent  on  elements  of  work  and 
nonwork  not  charted  as  activities. 


Exhibit  4  is  a  summary  of  all  work  and  nonwork  elements  for  the  time 
during  the  work  sampling  study.  The  study  observes  4  catalogers  working  on 
cataloging  activities  8  hours  a  day  for  3  days.  The  total  time  on  the  job 
was 


(4  men)  (8  hrs/day)  (3  days)  5760  man-minutes. 


Since  the  operators  were  completing  the  operator's  data  sheet  (see  Form  2)* 
during  the  study,  the  total  time  on  the  job  cannot  be  considered  representative 
of  a  typical  3-day  work  period.  In  order  to  derive  a  representative  total 
time  on  the  job  the  GAME  study  time  (see  Exhibit  4)  is  subtracted  from  the 
total  time  on  the  Job. 


Representative  total  time  on  the  job  ■  5760-360 

-  3400  man-minutes 


In  order  to  determine  if  the  present  method  should  be  performed  more  ef¬ 
ficiently  we  shall  scrutinize  the  amount  of  time  consumed  in  each  work  and 
nonwork  element  not  charted  on  the  flow  chart. 


An  index  above  .75  can  be  considered  adequate,  based  on  the  assumption  that 
152  of  staff  time  is  sufficient  for  personal,  fatigue  and  delay  allowances, 
more  than  52  instruction  time  is  excessive  and  more  than  52  transport  time 
is  excessive.  These  allowances  are  considered  reasonable  in  production 
plants  where  fatigue  factors  and  safety  hazards  are  low  and  where  the  en¬ 
vironment  is  good.  The  actual  allowance  should  be  set  bv  the  library 
administrator  at  each  library;  however,  as  a  base  line  for  efficiency 
measurement,  252  is  a  reasonable  allowance. 

Forms  1,2,3  and  4  are  provided  for  reproduction  purposes  and  are  on  pages 
25,  80,  81  and  82  respectively. 
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Exhibit 


The  percent  of  time  spent  on  the 

charted  activities  -  (4050) (1002) /5400  -  75.01 


The  percent  of  time  SDent  in  tnnsporting 
(between  activities)  ■  (480) (lOCi) /5400  ■  8.82 

The  percent  of  time  spent  for  the 

operators'  personal  use  ■  (270) (100%) /5400  ■  5.0% 

The  percent  of  time  spent  in 
instruction  -  (300) (100%) /5400  -  5.6% 

The  percent  of  time  SDent  in 

unavoidable  delays  -  ( 300) (100%) /5400  -  5.6% 

The  sum  of  these  5  percentages  is  100%. 

The  percent  of  time  lost  due  to  slow  pace 

(4050-3840) (100%)  (210) (100%) 

5400  "  5400  * 


The  total  utilization  for  the  period  studied  is  computed  by  the  percent  of 
time  spent  on  the  charted  activities  minus  the  percent  of  time  lost  due  to 
slow  pace.  For  the  example  above  total  utilization  of  the  cataloging  group 
-  75.0%-3.9%  -  71.1%  . 

Since  the  goal  is  to  attain  75%  utilization  or  better,  3.9%  should  be 
gained  by  eliminating  inefficiencies  in  the  present  method. 

The  GAME  Analvst  is  in  a  position  to  answer  kev  question  *1 ,  part  (a) - 

are  nonwork  element  times  in  excess  of  allowances** due  to  poor  management 
of  flow,  personnel  or  training?  The  studv  indicates  that  transporting 
materials  to  a..d  from  the  cataloging  area  takes  8.8%  of  the  catalogs-  rs ’  time. 
Since  transporting  time  greater  than  5%  is  considered  excessive,  we  should 
attempt  to  reduce  this  time.  A  poilcv  to  move  books  fror  receiving  to 
cataloging  and  from  cataloging  to  rther  processing  onlv  once  a  dav  mav  he 
sufficient  to  reduce  the  excessive  transport  time.  Other  transporting 
activities  should  be  investigated  to  determine  if  movement  between  work  areas 
can  be  reduced. 


The  utilization  of  71.1%  is  derived  from  the  3-  dav  work  sampling  studv. 
The  index  of  productive  ;  .702  is  derived  from  a  comparison  of  one  week 
of  work  with  a  standard.  Confidence  in  the  analysis  improves  as  the 
difference  in  the  two  measures  approaches  zero. 

**  Aa  a  loose  rule  of  thumb,  time  lost  in  slow  pace  bv  fatigue  and  all  other 
causes  should  not  exceed  5%.  Transport  lime,  personal  tine,  instruction 
time  and  unavoidable  delav  time  should  not  exceed  5%  each. 
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Further  improvement  in  the  efficiency  of  cataloging  may  be  realized 
by  reducing  instruction  time  through  a  comprehensive  training  program. 
Unavoidable  delay  time  which  is  Incurred  because  of  unbalanced  work 
stations  or  cyclical  work  loads  can  often  be  minimized  by  systematic 
production  controls.  If  unavoidable  delay  time  accounts  for  as  much  as 
10Z  or  more  of  total  staff  time  for  book  catalogers  or  book  processors 
and  if  library  management  is  unable  to  reduce  such  delays  to  5Z  or  below, 
professional  consultation  should  be  considered  for  larger  libraries. 

Excessive  personal  time  is  another  factor  to  ch  ck  in  determining  if 
the  existing  method  should  be  performed  more  efficiently  by  managing  it 
more  effectively.  If  personal  time  is  excessive  fHe  following  actions 
should  be  considered: 

(1)  Establish  set  break  periods. 

(2)  Require  a  higher  level  of  production  for  the  group  by 
reducing  man-hours  assigned  to  the  activity. 

(3)  If  (1)  and  (2)  prove  unsuccessful,  it  may  be  necessary  to 
encourage  group  pressures  on  the  most  serious  offenders  by 
discussing  the  problem  with  the  group  leaders. 

Step  #2  answers  key  question  #1,  part  (b) - is  the  average  pace  for 

the  group  below  the  expected  pace  and  not  due  to  factors  intrinsic  in  the 
method? 

If  average  pace  is  more  than  5Z  below  the  expected  level,  that  is. 
if  average  pace  is  lower  than  93Z  the  following  actions  should  be  cons  lered: 

(1)  Require  a  higher  level  of  production  for  the  group  by  reducing 
man-hours  assigned  to  the  activity. 

(2)  If  (1)  is  implemented  and  the  pace  regains  slow,  a  backlog 
will  most  likelv  occur.  Pressure  on  the  group  to  clear  up  the 
backlog  may  improve  the  average  pace  of  the  group. 

(3)  Various  incentive  programs  reav  be  implemented  to  improve  the 
pace,  such  as  giving  priority  for  advancer: ftt  and  benefits 
according  to  productivity. 

(4)  Various  programs  to  improve  the  pace  mav  be  Implemented, 
such  as  equipment  and  environmental  Improvements  to  reduce 
fatigue  and  boredom  and  improve  mrrale. 
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Step  if 3  determines  if  the  present  method  should  be  improved  to 
eliminate  excessive  personal  time,  transport  time,  instruction  time, 
unavoidable  delay  time,  etc.,  or  to  perform  the  operation  at  less  cost 
per  unit  of  work. 

Assume  that  the  work  sampling  3tudy  shows  that  30%  of  total  time 
lost  is  caused  by  personal  time,  transport  time,  instruction  time  and 
unavoidable  delays  between  activities  B2  and  E2  (see  Figure  14).  Assume 
that  further  investigation  shows  that  this  time  is  lost  due  to  queuing 
at  the  Library  of  Congress  proof  slip  file  and  the  National  Union  Catalog 
shelves,  that  is,  in  some  cases  several  catalogers  are  searching  or  wait¬ 
ing  to  search  for  a  precedent  at  one  time.  This  condition  is  conducive 
to  increased  personal  time,  instruction  time  and  avoidable  and  unavoidable 
delays.  Assume  that  further  study  indicates  that  20%  of  the  titles  searched 
were  published  within  the  last  6  months  and  that  only  5%  of  these  titles 
have  a  precedent.  Also  assume  that  50%  of  the  titles  searched  were  pub¬ 
lished  within  the  last  18  months  and  that  25%  of  these  titles  have  a 
precedent. 

If  a  policy  not  to  search  for  precedents  for  titles  published  within 
6  months  were  adopted,  a  savings  in  search  time  would  be  realized, 
perhaps,  for  example  7.5  min.  per  title.  If  the  policy  were  implemented 
for  titles  18  months  old,  assume  a  savings  of  4.0  min.  per  title  realized 
in  Library  of  Congress  precedent  searching.  However,  if  these  titles  were 
not  searched,  it  would  be  necessary  to  do  original  cataloging  on  all  of 
them.  Since  original  cataloging  requires  more  time  than  precedent 
cataloging,  say,  for  example  17.3  min.  per  title  more,  every  title  which 
had  a  precedent  will  require  17.3  min.  more  to  catalog  than  if  the 
precedent  were  searched  and  found. 

In  this  situation  break-even  points  should  be  calculated  to  determine 
the  percent  of  precedents  required  to  be  found  in  each  title  age  group 
to  make  the  search  worth  ”hile. 

Designate  Break  Even  Point  bv  (BEP). 

( B F 1* )  »  c *mc  t °  search  the  average  title  in  the  age  group _ 

(original  cataloging  time/titleHprecedent  cataloging  time/title) 

For  example: 

For  the  6  months  age  group,  BFP^  ■  -  .43 

4  0 

lor  the  18  months  age  group,  BEPjg  *  *  .23 

4  8 

For  the  12  months  age  group,  BEP12  *  YT~3  * 

* 

A  title  ace  grot  is  defined  as  a  group  of  titles  published  iater  than 
a  given  date. 
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To  find  the  age  group  which  should  not  be  searched,  the  locus  of 
points  of  the  percent  of  precedents  found  (PPF)  for  each  age  group  and  the 
locus  of  each  age  group’.-,  break  even  point  (BEP)  should  be  plotted.  Figure  15 
shows  the  plot  of  the  BEP^  locus  and  the  PPFj  locus,  where  1  can  be  any  age 
group. 

The  point  at  which  the  PPFi  locus  and  8EPj  locus  intersect*  will 
determine  the  age  group  which  should  not  be  searched.  Figure  15  indicates 
that  the  age  group  17.6  months  or  later  should  not  be  searched. 

A  policy  should  be  implemented  not  to  search  for  precedents  for  titles 
less  than,  e.g.,  17  months  old,  unless  by  previous  knowledge  the  decision 
maker  knows  that  there  is  a  precedent  available  or  that  there  is  a  high 
probability  of  finding  a  precedent  for  a  given  title. 

Since  a  decision  maker  is  required,  we  shall  change  the  method  by 
adding  an  activity.  This  activity  involves  a  decision  on  cataloging 
strategy.  This  activity  is  represented  in  Figure  16**as  cataloging 
strategy  decision  (B2'). 

The  new  method  is  expected  to  reduce  personal  time,  transport  time, 
instruction  time  and  delav  time  at  the  National  Union  Catalog  shelves  and 
at  the  Library  of  Congress  proof  slip  file.  Furthermore,  the  new  method  is 
expected  (1)  to  reduce  the  average  time/title  in  searching  for  a  precedent, 
(2)  to  reduce  the  percentage  of  titles  for  which  the  search  fails  to  ac¬ 
complish  the  objective  cf  locating  a  precedent  and  (3)  to  improve  the  flow 
of  materials  by  reducing  the  bottleneck  effects  below  that  of  the  old  method. 

The  above  example  in  step  // 3  is  only  one  kind  of  methods  analysis.  This 
example  serves  only  to  demonstrate  how  methods  may  he  changed  to  improve 
efficiency. 

TASK  5 

Task  5  reviews  the  find  .gs  of  the  previous  4  tasks  and  recommends 
policies  and  procedures  to  be  implemented  to  improve  the  efficiency  of  the 
operation,  thus  completing  the  GAME  Analysis. 


This  example  assumes  that  the  catalogers  are  doing  the  searching  and  that 
the  cost  per  minute  in  searching  is  the  same  as  the  cost  per  minute  in 
cataloging.  In  a  case  where  the  costs  are  not  the  same  then: 


KfP  ,  (Ws)(_tirae  to  search  the  average  title  in  the  age  group) _ _ _ _ 

(Woe) (Original  cataloging  time/ti tleHWpcTfprecedent  cataloging  time/title) 

where:  Ws  -  searcher’s  wage  rate,  Woe  *  original  cataloger's  wage  rate  and 
Wpc  *  precedent  cataloger's  wage  rate* 


Figure  16  is  a  flow  chart  of  the  proposed  method.  The  cataloging  activities 
effected  bv  the  change  are  the  only  activities  charted. 
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Figure  16 
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VIII.  APPLICATION  AND  RELIABILITY  OF  SCORE,  SCOUT,  CORE  AND  GAME  ANALYSES 


The  majority  of  the  time  spent  during  the  library  visits  of  Phase  III 
was  for  the  purpose  of  determining  the  feasibility  of  collecting  data 
essential  to  performing  analyses  of  library  efficiency  and  effectiveness 
by  using  the  tentative  methods  recommended  in  the  Phase  II  report. 

This  chapter  represents  the  findings  and  discusses  the  applicability 
of  methods  used  in  SCORE,  SCOUT,  CORE  and  GAME  Analyses  according  to  library 
size  and  types  of  library  operations  and  services. 


>licability  &  Reliability  of  SCORE  Analysis 


Sample  data  for  SCORE  Analysis  were  collected  at  three  A.T.L.'s. 
Reliable  estimates  of  the  percentage  of  needs  passing  several  events  in 
reference  search  service  (librarian  conducts  search)  were  derived  from  a 
study  of  a  total  of  69  or  fewer  needs  for  the  service  at  two  A.T.L.'s. 
The  percent  of  needs  passing  each  event  based  on  a  sample  size  of  69  or 
fewer  needs  entering  the  event  was  found  to  predict  the  probability  of 
occurrence  of  several  events  within  ±  102,  952  of  the  time. 


Table  5  shows  the  data  collected  at  one  library  for  determining  the 
effectiveness  probabilities.  We  shall  refer  to  this  library  as  library  #1. 
The  percent  of  occurrence  for  event  1  for  period  1  is 

||  (100)  -  972. 

The  percent  of  occurrence  for  event  1  for  period  2  is  ; 

||  (100)  -  1002. 


Table  6  shows  the  percent  of  occurrence  of  each  event  for  the  two  periods. 


If  for  a  given  event  the  percent  of  occurrence  for  different  periods  is 
relatively  stable,  the  probability  of  event  occurrence  can  be  determined. 

This  probability  predicts  the  percent  of  occurrence  of  the  event.  At 
library  Cl  the  total  number  of  needs  processed  during  the  5  day  studv  was  69. 
The  average  percent  of  occurrence  of  event  1  for  the  5  day  period  is: 

||  (100)  -  98.52. 

The  probability  of  event  occurrence  is  expressed  as  the  fractional  part  of 
the  needs  passing  the  event.  For  library  #1,  the  probability  of  event  1 
occurrence  is. 
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A  minimum  level  of  confidence  is  required  of  the  average  percent  of 
occurrence  of  each  event  before  the  probability  of  event  occurrence  can 
be  expressed. 

The  purpose  of  the  reliability  tests  in  Phase  III  was  to  determine 
the  study  time  required  in  order  to  derive  event  probabilities  with  a 
confidence  of  i  102,  952  of  the  time  at  the  sample  libraries. 

The  following  equation  was  used  to  determine  the  number  of  2-1/2*  day 
periods  to  be  studied  to  attain  the  required  confidence  in  the  probability 
of  occurrence  of  the  event  (i),  where  i  can  be  any  event: 

N.  r;ow(M)~|2 

1  Ld2  -  *1  J 


where:  Nj  »  number  of  2-1/2  day  study  periods  required, 

R  *  range  of  percent  of  occurrence  values  between 
the  period  of  highest  occurrence  and  the 
period  of  lowest  occurrence, 

N  *  the  number  of  2-1/2  day  periods  studied, 

d2  ■  a  constant  for  a  given  S  value  (see  Table  4), 

1'X^  ■  the  sum  of  percent  of  occurrences  of  event  i 
for  each  period. 


Nj  for  event  1  at  library  1  is: 


(20) (3) (2)  1  2  ^  [  120  1  2 
(1 .128X197)  J  “  [  222 


(.54)2. 


.29 


Since  Nj 


is  smaller  than  N  ,  no  additional 


2-1/2  day  periods  should  be 


studied.  That  is,  the  probability  .985  is  accurate  within  t  102,  952 
of  the  time,  as  a  predictor  o»  the  percent  of  needs  for  which  event  1 
occurs  for  any  5  day  period. 


2-1/2  days  was  us^d  as  the  time  Interval  for  the  library  studied.  The 
time  interval  depends  on  the  rate  of  incoming  needs.  The  time  interval 
chosen  should  include  at  least  25  incoming  needs. 
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N2  (for  event  2)  at  library  1  is: 

■  ■  f  (20)  (16)  (2)  I2  f640  /  in\2 

N2  [(1. 128)(184)  J  [207  J  ^3'107 

ti'2  -  9.6 

I 

Since  Nj  is  larger  than  N2,  additional  periods  must  be  studied  to  get 

the  probability  of  event  5.  After  each  additional  period  n'  should  be 
computed  to  determine  if  the  required  confidence  in  the  average  percent  of 
occurrence  has  been  reached  to  permit  expression  of  the  event  probability. 

The  5  day  study  at  library  #1  failed  to  generate  enough  data  to  ex¬ 
press  the  probability  of  events  9 2  and  #5.  The  n|  calculations  indicated 

that  24  day  study  of  occurrences  of  event  3  is  required;  24  days  is  the 
longest  period  required  to  derive  anv  event  probability  for  the  service  at 
library  #1.  Therefore,  it  is  predicted  that  it»  a  period  of  approximately 
one  month  the  Drobabllities  of  occurrence  of  all  events  can  be  derived  for 
reference  search  service  at  library  #1.  Libraries  which  have  fewer  than 
10  needs  per  day  in  reference  search  service  may  require  longer  studies. 

For  the  second  library  (library  #2  data  were  generated  over  a  2 
months  period,  and  all  the  event  probabilities  were  derived  with  the  desired 
confidence.  Thi3  smaller  library  has  a  staff  size  of  7.  Assuming  library 
#2  is  representative  of  its  staff-size  group  in  the  amount  of  reference 
search  service  given,  the  study  indicates  that  SCORE  Analysis  effectiveness 
measures  with  the  desired  confidence  can  be  derived  at  libraries  with  a 
staff  size  of  7  or  more  within  approximately  2  months. 

In  small  libraries  the  time  required  to  derive  event  probabilities, 
with  a  confidence  of  t  10Z,  95X  of  the  time,  mav  be  prohibitive.  Small 
libraries,  in  this  case,  are  defined  as  those  which  process  reference  search 
service  needs  at  an  average  rate  on  the  order  of  3  per  day  or  less.  If  less 
confidence  is  expected  and  more  time  is  allowed,  reference  search  service 
can  be  measured  bv  SCORE  Analysis  even  at  smaller  libraries. 

Data  for  effectiveness  measurement  of  users  search  service  bv  SCORE 
Analysis  were  collected  at  a  third  A.T.L.  (library  #3).  The  data  were 
generated  over  en  H  work  dav  period.  The  probabilities  of  event  occur¬ 
rence  were  derived  with  the  required  confidence  during  this  period.  The 
data  for  percent  of  occurrence  of  events  were  collected  bv  questionnaires 
given  to  the  users  of  the  service.  The  ‘eaponse  in  returning  the  queetion- 
nalres  was  poor  (approximately  43Z),  This  poor  response  tends  to  bias  the 
sample;  however,  regardless  of  the  bias  we  can  reasonably  assume  that  better 
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means  of  getting  user  feedback  and  a  study  of  up  to  20  work  days  would 
make  SCORE  Analysis  of  user  search  service  feasible  at  all  libraries 
with  a  staff  size  of  A  or  more.  If  less  confidence  and/or  more  time  could 
be  allowed  for  the  study,  I t  is  feasible  that  SCORE  Analysis  effectiveness 
measurements  of  user  search  service  could  be  conducted  at  all  A.T.L.'s. 

A  complete  SCORE  Analysis  requires  the  generation  of  cost  data.  Since 
library  costs  are  relatively  stable  for  labor  and  acquisitions,  there  is 
a  high  probability  that  reliable  cost  data  can  be  determined  within  the 
time  required  to  determine  reliable  effectiveness  measures.  Further 
discussions  of  costs  reliability  will  be  presented  in  Section  D  of  this 
chapter. 

B.  Applicability  and  Reliability  of  SCOUT  Analysis 

Tests  were  conducted  at  A  A.T.L.’s  to  determine  the  feasibility  of 
measuring  the  effectiveness  of  services  and  operations  with  methods  which 
depend  upon  the  use  of  librarians'  subjective  judgments  as  to  utility 
values  of  the  services  and  operations.  These  tests  are  applications  of 
utility  analysis  as  described  in  the  Phase  II  report.  We  will  undertake 
here  to  test  the  reliability  of  these  methods  in  two  respects: 

Test  1.  The  extent  to  which  utility  judgments  of  a  given  service 

or  operation  made  by  one  librarian  agree  with  the  judgments 
made  independently  by  another  librarian. 

Test  2.  The  extent  to  which  the  utility  judgments  made  by  a  librarian 
agree  with  his  judgments  of  how  funds  should  be  allocated. 

The  reliabilitv  of  the  utility  analyses  in  test  1  was  not  considered 
high  enough  to  recommend  the  use  of  utility  analysis  using  subjective 
judgments  without  an  objective  basis  for  the  judgments.  The  reliabilitv 
of  the  utility  analysis  in  test  2  was  highly  significant. 

At  each  of  the  A  libraries  the  utility  analysis  indicated  that,  in 
order  to  maximize  utility  of  the  services,  certain  changes  should  be  made 
in  the  allocation  of  funds.  At  3  of  the  A  libraries  the  librarians  agreed 
that  these  changes  in  budgeting  should  be  made  to  improve  effectiveness. 

In  effect,  test  2  showed  that  for  5  out  of  the  6  librarians  tested  their 
subjective  Judgments  of  utility  and  their  budgeting  criteria  were  con- 
siscent.  However,  because  in  test  1  the  reliabilitv  was  not  significant, 
we  doubt  the  feasibility  of  using  a  purely  subjective  utility  analysis. 

To  improve  the  credibilllv  and  reliabilitv  of  utility  analvsis  the 
original  concept  has  Incorporated  some  of  the  validated  objective  measures 
of  effectiveness  utilized  in  SCORE  Analvsis  to  form  SCOUT  Analvsis. 
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SCOUT  Analysis  uses  a  base  for  utility  judgments.  This  base  of 
utility  for  services  is  the  average  number  of  needs  met  bv  a  service  in 
a  given  period  of  time  with  objective  leveling.  The  base  of  utility  for 
operations  is  derived  from  the  change  in  the  number  of  needs  met  ^M)  due 
to  a  c'  ange  in  the  operations  output  AO,  i.e.,  base  utilitv  for  operations 
=  AM/ AO  .  The  reliability  of  this  measure  has  been  established  as  described 
in  Section  A  of  this  chapter.  Reliable  estimates  of  the  change  in  utility 
due  to  changes  in  operation  outnut  c.;n  be  derived  by  studies  of  previous 
needs  met  and/or  not  met.  For  example,  if  an  average  of  100  needs  in  refer¬ 
ence  search  service  are  met  per  week  and  if  an  average  of  10  of  these  needs 
are  met  through  4  man-hours/week  f  interlibrary  loan  activities,  a  change 
in  interlibrary  loan  activity  from  4  man-hours/veek  to  0  man-hrs/week  would 
cause  the  average  number  of  needs  met  per  week  to  change  from  100  to  90. 

The  — —  measurement  concept  of  SCOUT  Analysis  is  based  upon  one  of 

the  same  data  elements  as  the  effectiveness  measurement  of  SCORE  Analysis 

A  M 

is  based  upon,  that  i3,  the  number  of  needs  met.  Therefore,  is  as 

m  u 

reliable  for  measuring  base  utility  as  is  the  SCORE  delta  index  for  measur¬ 
ing  cost-effectiveness.  A  confidence  of  1  10%,  95%  of  the  Lime^should  be 
A  M 

attainable  for  by  a  study  of  approximately  the  same  duration  as  is 

requi-ed  to  perform  SCORE  Analysis. 

SCOlf  Analysis  has  been  introduced  to  account  for  the  value  of  needs 
met,  and  the  raw  number  of  needs  met  is  not  3  valid  measure  of  the  kind  of 
effectiveness  denoted  by  the  definitions  of  value  and  utilitv.  For  example, 
it  is  conceivable  that  the  typical  need  met  iri  reference  search  service  has 
a  greater  value  than  the  typical  need  met  in  predetermined  circulation  of 
periodicals.  SCOUT  Analysis,  therefore,  allows  for  leveling  of  the 
A  M 

measures  according  to  the  judgment  of  a  qualified  SCOUT  Analyst.  The 

credibility  of  these  judgments  cannot  at  this  time  be  supported  by  measures 
of  statistical  reliability;  however,  this  fact  should  not  be  used  to  dis¬ 
count  the  validity  of  the  approach. 

Section  D  of  this  chapter  discusses  GAME  Analysis  which  utilizes 
Group  Attainment  Program  (GAP).  GAP  is  a  technique  which  has  received  wide 
acceptance  as  a  means  for  measuring  manufacturing  efficiency.  GAF  uses 
a  leveling  principle  to  adjust  the  amount  of  time  spent  on  an  operation  to 
account  for  the  operator's  pace  of  working.  Regardless  of  the  subjective 
nature  of  leveling  ,it  is  considered  to  be  a  valid  criterion  for  measuring 
group  performance.  If  leveling  factors  are  cautiously  applied,  the  validity 
of  the  overall  measurement  of  efficiency,  effectiveness,  value  or  utility 
can  be  improved.  By  cautious  application  we  mean  that  an  analyst  should 
not  level  the  time  on  an  operation  or  the  base  utility  beyond  his  confidence 
or  ability. 
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As  the  analyst  gains  experience  he  will  be  able  to  level  more 
severelv.  For  example,  a  beginning  SCOUT  Analyst  should  not  level  refer¬ 
ence  search  and  circulation  service  as  0.5  times  the  value  of  user  search 
and  circulation  service  without  reliable  data  to  support  such  a  severe 
leveling  factor.  If  the  analyst's  study  shows  that  the  typical  user 
search  need  met  has  twice  the  effect  that  the  typical  reference  search 
need  met  has  upon  increasing  or  decreasing  the  solution  probability  of 
approaches  to  meeting  the  user  mission,  he  can  reliable  level  reference 
search  service  needs  at  0.5  (X),  where  X  is  the  leveled  utility  base  per 
need*  io  reference  search. 

To  summarize,  SCOUT  Analysis  is  a  feasible  method  for  measuring  library 
efficiency  and  effectiveness  and  is  recommended  for  use  by  trained  analysts 
at  all  A.T.L.'s  for  all  operations  and  services. 

C .  App li  cab i 1 ity  and  Reliability  of  CORE  Analysis 

CORE  Analysis  is  designed  to  develop  standards  for  efficiency  at  given 
levels  of  effectiveness.  The  standards  are  based  on  norms  for  a  group 
performing  within  a  definable  range  of  effectiveness  or  quality.  The 
standard  for  a  quality  range  will  be  the  expected  level  of  performance,  that  is, 
the  quality  should  not  range  below  the  lower  limit  of  the  defined  quality 
range.  Furthermore,  the  unit  cost  should  not  be  higher  than  the  standard 
cost  per  unit,  unless  the  quality  can  be  demonstrated  to  be  proportionally 
high  to  justify  the  additional  unit  costs  or  unless  extenuating  circum¬ 
stances  prevail. 

Basically  CORE  Analysis  is  a  means  for  identifying  areas  where  opera¬ 
tional  performance  standards  for  a  population  of  libraries  are  feasible, 
and  for  providing  a  tool  for  developing  such  standards.  The  measure  of 
performance  is  derived  by  comparison  of  actual  unit  costs  and  quality  to 
standard  unit  costs  and  quality. 

The  applicability  of  CORE  Analysis  depends  upon: 

(1)  the  ability  of  the  analyst  to  define  narrow  classes  of  quality; 

(2)  the  ability  of  the  analyst  to  define  quality  classes  in  such  a 
manner  that  the  cost-output  correlations  of  the  members  in  these 
classes  are  sufficiently  high;** 

(3)  the  extent  of  performance  conformity  among  the  libraries  being 
studied. 


The  leveled  utility  base  per  need  is  arbitrarily  set  at  10  for  a  middle 
ranking  service.  The  service  assigned  the  value  10  can  serve  as  a  bench 
mark  for  judging  the  leveled  utility  base  per  need  for  all  other  services. 

A  correlation  coefficient  of  0.80  should  be  considered  adequate  for  the 
development  of  tight  and  reliable  standards. 
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In  Phase  II,  correlation  analvses  were  conducted  on  a  variety  of 
data  classes.  Significant  curvilinear  correlations  were  found  between 
the  outputs  of  several  routine  operations  and  their  costs.  Conditional 
standards  for  varying  operational  outputs  can  feasibly  be  applied  to 
these  operations.  These  conditional  standards  should  express  unit  cost 
as  a  curvilinear  function  of  quantity  of  output.  CORK  Analysis  is 
designed  to  derive  linear  function  standards,  tiiat  is,  for  a  given  level 
of  effectiveness  the  CORE  standard  is  expressed  as  a  linear  function. 

For  example,  the  standard  for  descriptive  cataloging  nav  be  expressed 
bv  linear  equations  of  the  general  form  v  =  a  +  bx.  For  the  example  in 
Chapter  VI,  p.  58. 

v  *  -12.4  +  l.lx, 

where:  v  ~  weeklv  cost  of  descriptive 

cataloging , 

and  x  =  the  number  of  titles  descriptively 
cataloged  per  week. 

This  CORE  standard  will  be  applicable  for  a  limited  range  of  x  values, 
where  the  x  value  is  the  output  quantity.  If  this  standard  is  based  upon 
data  collected  from  10  libraries  performing  descriptive  cataloging,  where 
the  librarv  with  the  lowest  x  value  catalogs  20  titles  per  week  and  the 
library  with  the  highest  x  value  catalogs  224  titles  per  week,  the  stand¬ 
ard  will  anplv  only  to  libraries  cataloging  between  20  and  224  titles  per 
week.  Other  factors  should  be  considered  in  judging  the  applicability  of 
a  CORE  standard  such  as  the  following: 

(1)  The  validity  of  the  standard  improves  as  the  value  of  the 
constant  (a)  approaches  zero.  The  confidence  interval  for  v  is 

±  —  (100X)  of  v,  i.e.,  actual  cost  should  be  within  the  range 

^  3 

y  +  -  (y)  or  v  i  a. 

(2)  If  the  value  of  (a)  is  negative,  the  library  should  be  allowed 
to  deviate  from  the  standard  cost  v  by  a  factor  of  -$(a)  before  requiring 
the  administrator  to  account  for  cost  above  the  standard  cost,  that  is, 
if  a  =  -$20.00,  the  actual  cost  should  not  be  greater  than  y  -  $(a)  or 
y-(-$20.00)  or  y  +$20.00. 

(3)  Libraries  which  traditionally  work  below  the  standard  cost  should 
be  required  to  account  for  anv  recurring  drop  in  efficiency  (E)  .  For 
example,  if  a  library  is  measured  by  a  CORE  standard  for  January,  February 
and  March  and  E  »  ]  .20.  E  *  1.25,  E  =*  1.15,  respectively,  and  for  the 
months  of  April,  May  and  June  E  ■  1.00,  E  *  .95,  E  *  1.05,  respectively,  the 
apparent  drop  in  efficiency  during  April,  May  and  June  should  be  accounted 
for,  even  though  the  work  is  to  the  standard. 
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Based  upon  the  data  collected  in  Phase  II,  CORE  standards  are 
feasible  for  acquisitions,  cataloging,  circulation  activities  and 
bibliography  compilation  at  A.T.L.'s  with  staff  sizes  of  10  or  less. 

CORE  Analysis  is  not  a  feasible  tool  for  developing  standards  at  larger 
libraries,  because  it  has  been  observed  that  they  do  not  conform  to  the 
linear  equations  which  best  fit  the  smaller  libraries.  CORE  Analysis  is 
a  feasible  tool  for  developing  standards  for  large  groups  of  small  libraries 
which  are  characterized  bv  close  conformity.  The  libraries  with  staff  sizes 
over  10  do  not  constitute  a  large  group  and  do  not  conform  closely.  That 
is,  as  viewed  in  a  cost-output  scatter  diagram  these  larger  libraries  are 
few  and  far  between  (broadly  scattered)  .  The  feasibility  of  developing 
adequate  CORE  standards  decreases  as  the  scatter  broadens. 

Efficiency  standards  at  larger*  libraries  should  be  developed  by 
lAME  Analysis  (see  Chapter  VII). 

D .  Applicability  and  Reliability  of  CAME  Analysis 

A  test  conducted  during  Phase  III  at  one  A.T.L.  with  a  staff  size  of 
22  indicates  that  a  5-day  GAME  study  is  adequate  to  develop  reliable  stand¬ 
ards  with  a  confidence  level  of  +  10%,  95%  of  the  time,  for  cataloging 
activities  utilizing  A  or  more  full-time  group  members.  The  results  of 
this  test  support  the  feasibilitv  of  using  GAME  Analysis  "  r  developing  time 
and  cost  standards  for  routine  operations  at  large  libraries.  On  the  basis 
of  the  test  results  it  is  feasible  that  reliable  GAME  standards,  expressed 
as  expected  time/unit  or  cost/unit,  for  routine  operations  such  as  acquisitions, 
cataloging,  book  processing,  routine  reference  searches,  circulation  activ¬ 
ities  and  bibliographic  compilation  can  be  derived  for  large  libraries 
within  a  reasonable  study  time.  The  approximate  amount  of  time  required  to 
derive  a  reliable  standard  for  an  activity  can  be  predicted  by  a  preliminary 
GAME  Study.  This  is  done  by  first  computing  N'  (the  number  of  sample 
members  required  to  meet  the  desired  confidence  level),  see  d.  68  .  The 

id  step  is  to  determine  T  (the  time  required  to  study  one  sample  member). 

An  estimate  of  the  time  required  to  derive  a  reliable  standard  for  the 
activity  can  then  be  computed  bv  the  formula: 

studv  time  ■  N'  x  T. 

GAME  Analysis  can  be  used  more  practically  at  larger  libraries,  since 
in  general  the  study  will  require  less  time  and  expense.  Furthermore,  it 
is  probable  that  a  greater  savings  will  be  realized  from  the  study  at  larger 
libraries.  GAME  Analysis  is  to  he  used  prtmarilv  as  a  tool  for  measurement 
of  efficiency.  However,  effectiveness  of  the  outputs  of  operations  (quality 
of  outputs)  must  be  defined  and  controlled  in  GAME  Analysis.  The  measure  of 
effectiveness  in  GAME  Analysis  is  a  simple  quail  tv  check  to  determine  if  a 
predetermined  quality  standard  is  being  met. 


Larger  libraries  are  those  with  staff  sizes  of  11  or  more. 
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IX.  CONCLUSIONS 


This  contract  study  undertook  to  estiblish  criteria  for  the 
evaluation  of  Army  Technical  Library  operations  and  services.  In  terms 
of  the  study,  criteria  were  defined  as  concepts  usable  to  achieve  the 
objective.  In  this  study  the  contractor  is  charged  to  define  each 
criterion  which  is  recommended,  and  to  provide  a  method  of  implementing 
the  criterion  with  the  purpose  of  determining  library  effectiveness. 

In  carrying  out  the  work  of  the  study,  we  have  adhered  to  the  original 
division  of  the  project  into  three  phases.  In  the  first  phase  we  conducted 
a  thorough  search  of  the  literature  to  determine  the  state-of-the-art. 

This  search  was  aimed  not  only  at  locating  criteria  and  methods  which  have 
already  been  identified,  developed  and  applied  to  evaluation  of  library 
operations  and  services,  but  also  at  searching  out  techniques  outside  the 
field  of  library  sciences  which  could  be  applied  to  library  problems.  A 
very  considerable  number  of  such  references  were  located  in  both  cases. 
These  findings  were  reported,  along  with  certain  observations  as  to  the 
directions  the  work  should  take,  in  the  final  report  of  Phase  I. 

In  the  Phase  II  portion  of  the  study,  we  identified  a  number  of  dif¬ 
ferent  approaches  to  the  task  of  identifying  criteria,  and  sought  out  data 
and  information  by  which  these  approaches  could  be  analyzed.  These  data 
and  collections  of  information  were  sought  in  a  number  of  reports  of 
prior  statistical  studies  as  well  as  via  visits  to  a  selected  sample  of 
A.T.L.'s  by  personnel  of  John  I.  Thompson  &  Company.  From  the  experience 
gained  in  this  process  we  enunciated  a  number  of  candidate  criteria  and 
originated  certain  methods  or  techniques  which  we  deemed  adequate  for 
implementation  of  the  criteria  and  for  evaluating  library  operations  and 
services  performance  in  terms  of  efficiency  and  effectives. 

Phase  III  of  the  contract  study  was  devoted  to  examination  of  the 
criteria  and  development  of  detailed  Implementation  methods.  This  involved 
additional  visits  to  a  number  of  A.T.L.'s  for  discussions  with  library 
administrators,  for  gathering  certain  additional  data  and  information; 
for  developing  the  methods  into  systematized  procedures  and  applying  those 
procedures  to  library  situations.  Finally,  Phase  III  was  devoted  to 
detailing  these  procedures  into  definite  series  of  tasks  which  could  be 
followed  by  evaluation  personnel  in  applying  the  methods  in  specific 
library  cases. 

The  final  outcome  of  the  work,  therefore,  as  represented  by  this 
report,  is  a  number  of  criteria  which  represent  rather  fundamental  aspects 
of  the  operations  of,  and  the  services  and  products  produced  by,  A.T.L.'s. 
Accompanying  these  criteria  are  four  basic  procedures  which  we  might 
identify  as  management  techniques.  Actually,  several  classical  management 
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techniques  or  modifications  of  techniques  are  embodied  in  these  four 
systematized  procedures.  We  have  assigned  acronyms  to  these  procedures 
from  phrases  describing  the  actions  taken  in  the  system: 

SCORE  -  Service  Components  Reliability  and  Efficiency  Analysis 

SCOUT  -  Service  Components  UTility  Analysis 

CORE  -  Correlation,  Regression  and  Effectiveness  Analysis 

GAME  -  Group  Attainment  and  MEthods  Analysis. 

The  systematized  procedures  do  not  implement  all  the  criteria  which 
arose  out  of  the  work.  Nor  are  the  systematized  procedures  the  only  ones 
which  could  have  been  originated  and  developed.  However,  thev  do  embody 
what  we  judged  to  be  the  most  applicable  criteria  and  the  most  useful 
methods  for  development  at  the  present  time  with  the  data  available.  We 
believe,  moreover,  that  certain  additional  work  should  be  done  with  regard 
to  application  of  the  criteria  not  treated  finally  herein  and  the  develop¬ 
ment  of  management  procedures  which  are  available  at  the  present  time  for 
criteria  already  recognized. 

In  carrying  out  research  and  development  of  the  nature  of  this  contract 
work,  the  full  value  of  the  project  is  often  not  attained  because  the 
development  ceases  prematurely.  In  the  case  of  the  systematized  procedures 
developed  in  this  work,  thev  should  be  applied,  and  evaluators  familiarized 
with  them  in  detail,  before  their  real  worth  can  be  assessed.  It  Is  our 
recommendation,  therefore,  that  these  techniques  be  applied  in  3  sufficient 
number  of  technical  libraries  to  determine  through  usage  itself  whatever 
refinements  are  necessary  to  convert  them  to  standardized  practices. 

We  recommend  the  start  of  an  organized  effort  to  apply  the  methods 
at  a  sample  of  Army  Technical  Libraries.  We  further  recommend  that 
evaluators  be  trained  in  the  use  of  the  methods  and  that  the  methods  be 
applied  according  to  the  following  criteria: 

1.  For  measurement  of  efficiency  and  effectiveness  at  smaller 
libraries  (staff  size  of  10  or  less): 


1st  priority: 
2nd  priority: 
3rd  priority: 
4th  priority: 


SCOUT  Analysis 
SCORE  Analysis 
CORE  Analysis 
GAME  Analysis 


2.  For  measurement  of  efficiency  and  effectiveness  at  larger 
libraries  (staff  size  of  11  or  more): 


1st  priority: 
2nd  priority: 
3rd  priority: 


SCORE  Analysis 
SCOUT  Analysis 
GAME  Analysis. 
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